Jul 07 13:14:02.200894 ip-10-0-136-215 systemd[1]: Starting Kubernetes Kubelet... Jul 07 13:14:02.684866 ip-10-0-136-215 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 13:14:02.684866 ip-10-0-136-215 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 13:14:02.684866 ip-10-0-136-215 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 13:14:02.684866 ip-10-0-136-215 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 13:14:02.684866 ip-10-0-136-215 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 13:14:02.684866 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.624366 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 13:14:02.684866 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630165 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:14:02.684866 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630177 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:14:02.684866 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630181 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:14:02.684866 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630185 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:14:02.684866 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630188 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:14:02.684866 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630190 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:14:02.684866 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630193 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:14:02.684866 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630196 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630199 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630202 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630204 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630207 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630213 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630216 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630218 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630220 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630223 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630226 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630229 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630231 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630234 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630237 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630247 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630257 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630260 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630263 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630265 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:14:02.685662 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630268 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630270 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630273 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630276 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630296 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630305 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630308 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630310 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630313 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630316 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630318 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630320 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630323 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630327 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630329 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630332 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630335 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630337 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630340 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630343 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:14:02.686220 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630345 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630348 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630352 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630357 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630361 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630367 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630372 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630375 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630378 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630382 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630385 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630387 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630390 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630393 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630396 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630398 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630401 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630404 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630408 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:14:02.686953 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630411 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630414 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630417 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630419 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630422 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630425 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630427 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630430 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630432 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630435 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630437 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630440 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630442 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630445 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630447 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630451 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630453 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630456 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630459 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630462 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:14:02.687818 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630465 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630468 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630471 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630473 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630476 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630478 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630481 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630483 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630485 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630488 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630491 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630493 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.630496 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631189 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631196 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631199 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631202 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631206 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631209 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631211 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:14:02.688451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631214 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631217 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631220 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631223 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631226 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631229 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631232 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631235 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631239 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631242 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631246 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631250 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631253 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631257 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631260 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631263 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631266 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631269 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631272 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:14:02.689071 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631275 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631278 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631280 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631283 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631285 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631288 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631291 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631294 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631296 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631299 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631302 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631304 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631307 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631310 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631312 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631315 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631318 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631320 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631323 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631326 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:14:02.689907 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631328 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631330 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631333 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631336 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631338 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631341 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631344 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631346 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631349 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631351 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631356 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631359 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631362 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631364 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631367 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631369 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631372 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631375 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631377 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631380 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:14:02.690555 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631383 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631385 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631388 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631391 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631393 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631396 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631399 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631401 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631404 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631406 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631410 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631414 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631417 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631420 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631423 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631425 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631428 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631432 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631434 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631437 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:14:02.691183 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631439 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631442 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631446 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631448 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631451 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631453 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631456 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631458 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631461 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631464 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631467 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631469 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631486 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631846 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631851 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631854 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631857 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631860 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631862 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631865 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:14:02.691829 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631867 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631870 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631873 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631875 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631878 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631880 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631883 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631885 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631888 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631891 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631894 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631896 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631899 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631901 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631906 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631909 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631911 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631914 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631916 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631919 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:14:02.692451 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631922 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631924 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631927 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631929 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631932 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631935 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631938 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631941 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631943 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631946 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631949 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631954 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631957 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631959 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631963 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631966 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631968 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631971 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631974 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:14:02.693290 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631976 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631979 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631981 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631984 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631986 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631989 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631991 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631996 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.631998 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632001 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632003 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632006 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632008 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632011 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632013 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632016 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632019 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632021 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632024 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632026 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:14:02.693933 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632029 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632031 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632034 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632036 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632041 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632044 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632047 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632049 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632052 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632054 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632057 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632059 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632062 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632064 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632067 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632069 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632072 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632074 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632077 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:14:02.694821 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632081 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632083 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632086 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632089 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632091 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632094 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632096 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632099 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632101 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632105 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632109 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632112 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632115 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.632118 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632184 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632191 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632197 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632202 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632206 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632210 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 13:14:02.695893 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632214 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632218 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632221 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632224 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632227 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632230 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632233 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632236 2568 flags.go:64] FLAG: --cgroup-root="" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632239 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632242 2568 flags.go:64] FLAG: --client-ca-file="" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632245 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632248 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632253 2568 flags.go:64] FLAG: --cluster-domain="" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632255 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632259 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632261 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632265 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632268 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632271 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632274 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632277 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632280 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632283 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632286 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 13:14:02.696507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632289 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632293 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632297 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632300 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632302 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632306 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632309 2568 flags.go:64] FLAG: --enable-server="true" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632312 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632316 2568 flags.go:64] FLAG: --event-burst="100" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632319 2568 flags.go:64] FLAG: --event-qps="50" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632322 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632325 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632328 2568 flags.go:64] FLAG: --eviction-hard="" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632331 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632334 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632337 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632340 2568 flags.go:64] FLAG: --eviction-soft="" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632343 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632346 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632348 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632353 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632356 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632358 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632361 2568 flags.go:64] FLAG: --feature-gates="" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632369 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 13:14:02.697350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632372 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632375 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632378 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632381 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632384 2568 flags.go:64] FLAG: --help="false" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632387 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-136-215.ec2.internal" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632390 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632393 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632396 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632399 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632403 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632406 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632411 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632413 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632416 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632419 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632422 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632425 2568 flags.go:64] FLAG: --kube-reserved="" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632427 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632430 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632433 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632436 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632439 2568 flags.go:64] FLAG: --lock-file="" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632441 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 13:14:02.698160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632444 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632447 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632452 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632456 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632459 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632462 2568 flags.go:64] FLAG: --logging-format="text" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632464 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632468 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632470 2568 flags.go:64] FLAG: --manifest-url="" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632473 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632477 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632480 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632484 2568 flags.go:64] FLAG: --max-pods="110" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632487 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632489 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632492 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632495 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632498 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632501 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632504 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632511 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632514 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632517 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 13:14:02.699122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632520 2568 flags.go:64] FLAG: --pod-cidr="" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632523 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632528 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632531 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632534 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632537 2568 flags.go:64] FLAG: --port="10250" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632540 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632543 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d901c061a18a04de" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632546 2568 flags.go:64] FLAG: --qos-reserved="" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632549 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632552 2568 flags.go:64] FLAG: --register-node="true" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632555 2568 flags.go:64] FLAG: --register-with-taints="" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632560 2568 flags.go:64] FLAG: --registry-burst="10" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632563 2568 flags.go:64] FLAG: --registry-qps="5" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632565 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632568 2568 flags.go:64] FLAG: --reserved-memory="" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632584 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632588 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632591 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632593 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632596 2568 flags.go:64] FLAG: --runonce="false" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632599 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632602 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632605 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632608 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 13:14:02.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632611 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632614 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632617 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632974 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632977 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632982 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632985 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632988 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632991 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632994 2568 flags.go:64] FLAG: --system-cgroups="" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.632996 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.633002 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.633004 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.633007 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.633011 2568 flags.go:64] FLAG: --tls-min-version="" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.633013 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.633016 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.633019 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.633022 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.633027 2568 flags.go:64] FLAG: --v="2" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.633030 2568 flags.go:64] FLAG: --version="false" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.633034 2568 flags.go:64] FLAG: --vmodule="" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.633037 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.633041 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633143 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:14:02.804532 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633147 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:14:02.705935 ip-10-0-136-215 systemd[1]: Started Kubernetes Kubelet. Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633150 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633153 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633155 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633158 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633161 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633163 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633166 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633169 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633171 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633174 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633176 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633180 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633183 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633186 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633189 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633191 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633194 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633197 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:14:02.805780 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633199 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633202 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633204 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633206 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633209 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633213 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633217 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633221 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633224 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633226 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633229 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633232 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633234 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633237 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633239 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633242 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633245 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633247 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633250 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633252 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:14:02.806530 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633255 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633257 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633259 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633262 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633264 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633268 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633271 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633274 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633276 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633279 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633281 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633284 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633286 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633289 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633292 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633294 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633297 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633299 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633302 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633305 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:14:02.807304 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633308 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633310 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633314 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633317 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633320 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633323 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633325 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633328 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633330 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633333 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633335 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633338 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633340 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633343 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633345 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633348 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633350 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633355 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633358 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633360 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:14:02.807965 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633363 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633365 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633367 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633370 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633372 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633375 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633378 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633382 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633385 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633387 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633390 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633394 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633396 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633399 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633401 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633404 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633407 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633409 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:14:02.808688 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.633412 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.634030 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.647102 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.647118 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.647135 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.647141 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647201 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647206 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647210 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647213 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647216 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647219 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647222 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647225 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647228 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:14:02.809265 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647231 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647234 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647237 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647240 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647243 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647245 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647248 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647251 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647254 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647256 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647259 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647262 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647264 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647267 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647270 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647272 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647275 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647277 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647280 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647283 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:14:02.809865 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647286 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647288 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647292 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647294 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647297 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647299 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647302 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647305 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647308 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647310 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647313 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647316 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647318 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647321 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647324 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647326 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647329 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647331 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647334 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:14:02.810475 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647336 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647338 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647341 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647343 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647346 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647349 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647351 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647354 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647356 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647359 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647361 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647364 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647367 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647369 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647372 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647376 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647379 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647382 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647384 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647389 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:14:02.811132 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647391 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647394 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647396 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647399 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647401 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647404 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647408 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647412 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647415 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647418 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647421 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647430 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647433 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647435 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647438 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647440 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647443 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647445 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647449 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:14:02.811736 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647453 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:14:02.812342 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647456 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:14:02.812342 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647459 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:14:02.812342 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647461 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:14:02.812342 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647464 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:14:02.812342 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647467 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:14:02.812342 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647469 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:14:02.812342 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647472 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:14:02.812342 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647476 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:14:02.812342 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647491 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:14:02.812342 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647495 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:14:02.812342 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647498 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:14:02.812342 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.647505 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 13:14:02.812342 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647637 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:14:02.812342 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647643 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647646 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647649 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647652 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647655 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647658 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647660 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647663 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647665 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647668 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647671 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647679 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647682 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647685 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647687 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647689 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647692 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647694 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647697 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647700 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:14:02.812853 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647702 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647705 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647707 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647710 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647712 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647715 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647717 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647721 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647724 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647726 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647730 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647732 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647735 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647738 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647741 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647746 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647749 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647752 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:14:02.813487 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647755 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647759 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647762 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647764 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647767 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647770 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647773 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647776 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647778 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647781 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647783 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647786 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647789 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647791 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647794 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647796 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647799 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647801 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647803 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647806 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:14:02.814181 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647808 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647812 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647814 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647817 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647821 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647823 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647826 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647828 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647831 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647833 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647835 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647838 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647840 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647843 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647845 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647848 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647850 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647852 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647855 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647858 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:14:02.814884 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647860 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647863 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647865 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647868 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647870 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647873 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647875 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647877 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647880 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647883 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647885 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647887 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647890 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647893 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647896 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647899 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647902 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647905 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647908 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:14:02.815586 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:02.647910 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:14:02.816487 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.647915 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 13:14:02.816487 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.648032 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 13:14:02.816487 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.649946 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 13:14:02.816487 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.651296 2568 server.go:1019] "Starting client certificate rotation" Jul 07 13:14:02.816487 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.651395 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 13:14:02.816487 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.651434 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 13:14:02.816487 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.654608 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 13:14:02.816487 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.655725 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 13:14:02.816487 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.665554 2568 log.go:25] "Validated CRI v1 runtime API" Jul 07 13:14:02.816487 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.669303 2568 log.go:25] "Validated CRI v1 image API" Jul 07 13:14:02.816487 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.670090 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 13:14:02.816487 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.672321 2568 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9cc2e3cc-5a77-42d0-b5ea-edb6e94757d7:/dev/nvme0n1p4 f79b16b6-923f-42d6-aaad-1c05f368cd45:/dev/nvme0n1p3] Jul 07 13:14:02.817164 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.672339 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 13:14:02.817401 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.676544 2568 manager.go:216] Machine: {Timestamp:2026-07-07 13:14:02.676106667 +0000 UTC m=+0.354667990 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102699 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f11044a17a74ba6b93de48ccd9ee9 SystemUUID:ec2f1104-4a17-a74b-a6b9-3de48ccd9ee9 BootID:fd51999c-2535-4f50-a8b8-94cdc01cdb9a Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cf:c8:9d:98:c3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cf:c8:9d:98:c3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:17:9e:2f:5e:c3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 13:14:02.817401 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.676659 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 13:14:02.817401 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.676725 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 13:14:02.817401 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.680271 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 13:14:02.817401 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.680300 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-215.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 13:14:02.817401 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.681713 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 13:14:02.817401 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.681726 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 13:14:02.817401 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.681734 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.682536 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.683961 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.693985 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.694172 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.694209 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.694223 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.694251 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.694272 2568 kubelet.go:398] "Adding apiserver pod source" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.694302 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.694723 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.694747 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.695698 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.696824 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.698676 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.698703 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.700197 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.700210 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.700216 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.700222 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.700231 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.700236 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.700243 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.700249 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.700255 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.700260 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.700274 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.700279 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.702272 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.702298 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.705221 2568 server.go:1268] "Started kubelet" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.705557 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.706741 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.707817 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.707859 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:02.708675 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-215.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.708722 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-215.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:02.708764 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.708954 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 13:14:02.820374 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:02.715207 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.716560 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.716587 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:02.715393 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-215.ec2.internal.18c0042cc1340bc8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-215.ec2.internal,UID:ip-10-0-136-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-215.ec2.internal,},FirstTimestamp:2026-07-07 13:14:02.70520212 +0000 UTC m=+0.383763442,LastTimestamp:2026-07-07 13:14:02.70520212 +0000 UTC m=+0.383763442,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-215.ec2.internal,}" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:02.716727 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.717345 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.717348 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.717367 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.717400 2568 factory.go:55] Registering systemd factory Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.717416 2568 factory.go:223] Registration of the systemd container factory successfully Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.717437 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.717445 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.717637 2568 factory.go:153] Registering CRI-O factory Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.717651 2568 factory.go:223] Registration of the crio container factory successfully Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.717701 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.717726 2568 factory.go:103] Registering Raw factory Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.717741 2568 manager.go:1196] Started watching for new ooms in manager Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.718224 2568 manager.go:349] Starting recovery of all containers Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:02.720400 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:02.720554 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-215.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.727700 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9dhdm" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.732659 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9dhdm" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.733688 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.735366 2568 manager.go:354] Recovery completed Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.739528 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.743095 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.743123 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.743133 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.743618 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.743627 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.743641 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.743692 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.744525 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:14:02.821982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.744547 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.744555 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:02.745361 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-215.ec2.internal.18c0042cc3767703 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-215.ec2.internal,UID:ip-10-0-136-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-215.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-215.ec2.internal,},FirstTimestamp:2026-07-07 13:14:02.743109379 +0000 UTC m=+0.421670705,LastTimestamp:2026-07-07 13:14:02.743109379 +0000 UTC m=+0.421670705,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-215.ec2.internal,}" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.745521 2568 policy_none.go:49] "None policy: Start" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.745535 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.745544 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.747344 2568 policy_none.go:47] "Start" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.797861 2568 manager.go:337] "Starting Device Plugin manager" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:02.797900 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.797914 2568 server.go:87] "Starting device plugin registration server" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.798173 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.798340 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.798419 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.798504 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.798514 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:02.799082 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:02.799137 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.899190 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.902287 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.902319 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.902336 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.902371 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-215.ec2.internal" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.912305 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-215.ec2.internal" Jul 07 13:14:02.917315 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:02.912330 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-215.ec2.internal\": node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:02.950983 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:02.936974 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.964367 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.964403 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.964424 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:02.964453 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:02.967134 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:03.038119 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.066375 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal"] Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.066465 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.067507 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.067536 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.067546 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.069830 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.070004 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.070029 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.070593 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.070606 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.070625 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.070639 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.070643 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.070654 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.073268 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.074466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.073975 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:14:03.080278 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.075641 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:14:03.080278 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.075669 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:14:03.080278 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.075683 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:14:03.186734 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.119798 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1ee8660983e09b5aeea7a0e782f81de-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"a1ee8660983e09b5aeea7a0e782f81de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.186734 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.119831 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c861ca34efcd36b7887620391a65da0e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-215.ec2.internal\" (UID: \"c861ca34efcd36b7887620391a65da0e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.186734 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.119852 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a1ee8660983e09b5aeea7a0e782f81de-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"a1ee8660983e09b5aeea7a0e782f81de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.186734 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:03.121422 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-215.ec2.internal\" not found" node="ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.186734 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:03.126710 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-215.ec2.internal\" not found" node="ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.186734 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:03.138392 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:03.317232 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.220621 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c861ca34efcd36b7887620391a65da0e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-215.ec2.internal\" (UID: \"c861ca34efcd36b7887620391a65da0e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.317232 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.220648 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a1ee8660983e09b5aeea7a0e782f81de-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"a1ee8660983e09b5aeea7a0e782f81de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.317232 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.220668 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1ee8660983e09b5aeea7a0e782f81de-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"a1ee8660983e09b5aeea7a0e782f81de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.317232 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.220680 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c861ca34efcd36b7887620391a65da0e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-215.ec2.internal\" (UID: \"c861ca34efcd36b7887620391a65da0e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.317232 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.220707 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1ee8660983e09b5aeea7a0e782f81de-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"a1ee8660983e09b5aeea7a0e782f81de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.317232 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.220737 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a1ee8660983e09b5aeea7a0e782f81de-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"a1ee8660983e09b5aeea7a0e782f81de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.317232 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:03.238809 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:03.464757 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:03.339067 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:03.464757 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.424878 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.464757 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.430395 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 07 13:14:03.464757 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:03.439161 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:03.605302 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:03.539528 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:03.732314 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:03.640296 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:03.732314 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.651272 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 13:14:03.732314 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.651401 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 13:14:03.850515 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.735079 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 13:09:03 +0000 UTC" deadline="2028-03-05 13:06:12.746544201 +0000 UTC" Jul 07 13:14:03.850515 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.735108 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14567h52m9.011439805s" Jul 07 13:14:03.850515 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:03.740441 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:03.850515 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.796035 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:14:03.850515 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:03.841568 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:03.997799 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:03.942564 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:03.997799 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:03.960648 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:14:04.156187 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:04.043667 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:04.156187 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:04.144871 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 07 13:14:04.336748 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:04.191960 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ee8660983e09b5aeea7a0e782f81de.slice/crio-c36462fb48f4b9f50facf9f7c0ba382b6a7dc97842441fab23bf31417679feaa WatchSource:0}: Error finding container c36462fb48f4b9f50facf9f7c0ba382b6a7dc97842441fab23bf31417679feaa: Status 404 returned error can't find the container with id c36462fb48f4b9f50facf9f7c0ba382b6a7dc97842441fab23bf31417679feaa Jul 07 13:14:04.336748 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:04.192391 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc861ca34efcd36b7887620391a65da0e.slice/crio-9a5574c4ea7e2b8eec0f1e83b1b986b1d5031e032600f396cea288214b60e8ac WatchSource:0}: Error finding container 9a5574c4ea7e2b8eec0f1e83b1b986b1d5031e032600f396cea288214b60e8ac: Status 404 returned error can't find the container with id 9a5574c4ea7e2b8eec0f1e83b1b986b1d5031e032600f396cea288214b60e8ac Jul 07 13:14:04.336748 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.197070 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:14:04.336748 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.207514 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:14:04.336748 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.217246 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 07 13:14:04.336748 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.235244 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 13:14:04.336748 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.236643 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 07 13:14:04.336748 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.250029 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 13:14:04.697529 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.697424 2568 apiserver.go:52] "Watching apiserver" Jul 07 13:14:04.709496 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.709467 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 13:14:04.710711 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.710688 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87","openshift-image-registry/node-ca-z856v","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal","openshift-multus/network-metrics-daemon-6l86h","openshift-ovn-kubernetes/ovnkube-node-8wjmr","kube-system/global-pull-secret-syncer-j2rqx","openshift-cluster-node-tuning-operator/tuned-4gqpp","openshift-dns/node-resolver-q9ck9","openshift-multus/multus-additional-cni-plugins-h8rgd","openshift-multus/multus-twksx","openshift-network-diagnostics/network-check-target-7cw9x","openshift-network-operator/iptables-alerter-cm9xz","kube-system/konnectivity-agent-hm7dm"] Jul 07 13:14:04.714358 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.714341 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.716134 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.716114 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cm9xz" Jul 07 13:14:04.717356 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.717327 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 13:14:04.718238 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.718213 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z856v" Jul 07 13:14:04.720493 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.720476 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hm7dm" Jul 07 13:14:04.722863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.722652 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 13:14:04.722863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.722662 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 13:14:04.722863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.722756 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 13:14:04.722863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.722641 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 13:14:04.722863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.722855 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 13:14:04.723130 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.722933 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mlhzc\"" Jul 07 13:14:04.723130 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.723103 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:04.723194 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:04.723174 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:04.725190 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.725175 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.727162 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.727144 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q9ck9" Jul 07 13:14:04.729412 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.729392 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:04.729508 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:04.729469 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:04.731755 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.731735 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.731827 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.731742 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srktn\" (UniqueName: \"kubernetes.io/projected/28106265-9c28-4fe8-9678-40dc80bb00a8-kube-api-access-srktn\") pod \"iptables-alerter-cm9xz\" (UID: \"28106265-9c28-4fe8-9678-40dc80bb00a8\") " pod="openshift-network-operator/iptables-alerter-cm9xz" Jul 07 13:14:04.731885 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.731848 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0bbd3146-8a83-4cdd-9b41-410b53b540f4-host\") pod \"node-ca-z856v\" (UID: \"0bbd3146-8a83-4cdd-9b41-410b53b540f4\") " pod="openshift-image-registry/node-ca-z856v" Jul 07 13:14:04.731944 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.731881 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-cnibin\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.731944 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.731913 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.732019 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.731941 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.732019 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.731988 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0bbd3146-8a83-4cdd-9b41-410b53b540f4-serviceca\") pod \"node-ca-z856v\" (UID: \"0bbd3146-8a83-4cdd-9b41-410b53b540f4\") " pod="openshift-image-registry/node-ca-z856v" Jul 07 13:14:04.732105 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.732029 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-system-cni-dir\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.732105 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.732059 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-os-release\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.732105 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.732084 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-cni-binary-copy\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.732217 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.732109 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28106265-9c28-4fe8-9678-40dc80bb00a8-host-slash\") pod \"iptables-alerter-cm9xz\" (UID: \"28106265-9c28-4fe8-9678-40dc80bb00a8\") " pod="openshift-network-operator/iptables-alerter-cm9xz" Jul 07 13:14:04.732217 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.732131 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q78dq\" (UniqueName: \"kubernetes.io/projected/0bbd3146-8a83-4cdd-9b41-410b53b540f4-kube-api-access-q78dq\") pod \"node-ca-z856v\" (UID: \"0bbd3146-8a83-4cdd-9b41-410b53b540f4\") " pod="openshift-image-registry/node-ca-z856v" Jul 07 13:14:04.732217 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.732149 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.732217 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.732178 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x4x6\" (UniqueName: \"kubernetes.io/projected/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-kube-api-access-4x4x6\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.732217 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.732194 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28106265-9c28-4fe8-9678-40dc80bb00a8-iptables-alerter-script\") pod \"iptables-alerter-cm9xz\" (UID: \"28106265-9c28-4fe8-9678-40dc80bb00a8\") " pod="openshift-network-operator/iptables-alerter-cm9xz" Jul 07 13:14:04.733932 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.733911 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.735938 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.735918 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-twksx" Jul 07 13:14:04.738371 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.738356 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:04.738469 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:04.738411 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:04.753091 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.753064 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 13:14:04.753091 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.753077 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 13:14:04.753244 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.753104 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 13:14:04.753244 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.753183 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 13:14:04.753962 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.753944 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 13:14:04.754041 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.754012 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 13:14:04.754100 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.754013 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 13:14:04.754100 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.754046 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 13:14:04.754100 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.754073 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 13:14:04.754238 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.754122 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 13:14:04.755217 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.755200 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 13:14:04.755311 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.755204 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 13:14:04.755311 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.755263 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 13:14:04.755625 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.755566 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 13:14:04.755625 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.755567 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 13:14:04.755780 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.755572 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 13:14:04.755842 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.755816 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-drj2w\"" Jul 07 13:14:04.755896 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.755883 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 13:14:04.755949 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.755887 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lqnpt\"" Jul 07 13:14:04.755949 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.755922 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 13:14:04.755949 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.755934 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 13:14:04.755949 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.755946 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jbx84\"" Jul 07 13:14:04.756137 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.756054 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 13:14:04.756137 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.756065 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 13:14:04.756229 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.756136 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ms4lq\"" Jul 07 13:14:04.757350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.757330 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m5j59\"" Jul 07 13:14:04.757833 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.757807 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lpvts\"" Jul 07 13:14:04.758016 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.757995 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ct99h\"" Jul 07 13:14:04.758110 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.758051 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m5cx6\"" Jul 07 13:14:04.758473 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.758453 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 13:14:04.800168 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.800142 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 13:14:04.819020 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.818998 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 13:14:04.832807 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.832780 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-tuned\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.832927 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.832811 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-run-netns\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.832927 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.832816 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tjk58" Jul 07 13:14:04.832927 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.832832 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-multus-conf-dir\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.832927 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.832852 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-etc-openvswitch\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.832927 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.832871 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-var-lib-kubelet\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.832927 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.832894 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-system-cni-dir\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.832927 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.832915 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-run-k8s-cni-cncf-io\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.833183 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.832936 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs\") pod \"network-metrics-daemon-6l86h\" (UID: \"f63deb5a-7f0d-4cca-a94a-3f3e0892379b\") " pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:04.833183 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.832984 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-node-log\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.833183 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833015 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fbee5bee-337d-40af-baef-deee6657a83a-ovnkube-config\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.833183 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833074 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfh7m\" (UniqueName: \"kubernetes.io/projected/fbee5bee-337d-40af-baef-deee6657a83a-kube-api-access-gfh7m\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.833183 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833099 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/730e3c49-83b7-4a1d-9b7c-d5621955f6df-kubelet-config\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:04.833183 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833129 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0bbd3146-8a83-4cdd-9b41-410b53b540f4-host\") pod \"node-ca-z856v\" (UID: \"0bbd3146-8a83-4cdd-9b41-410b53b540f4\") " pod="openshift-image-registry/node-ca-z856v" Jul 07 13:14:04.833183 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833168 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-multus-cni-dir\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.833416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833197 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-run-multus-certs\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.833416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833215 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0bbd3146-8a83-4cdd-9b41-410b53b540f4-host\") pod \"node-ca-z856v\" (UID: \"0bbd3146-8a83-4cdd-9b41-410b53b540f4\") " pod="openshift-image-registry/node-ca-z856v" Jul 07 13:14:04.833416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833247 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fbee5bee-337d-40af-baef-deee6657a83a-env-overrides\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.833416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833264 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/730e3c49-83b7-4a1d-9b7c-d5621955f6df-dbus\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:04.833416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833298 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zxrv\" (UniqueName: \"kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv\") pod \"network-check-target-7cw9x\" (UID: \"6037cecb-d996-4bd8-a3d3-733a764f5163\") " pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:04.833416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833327 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0bbd3146-8a83-4cdd-9b41-410b53b540f4-serviceca\") pod \"node-ca-z856v\" (UID: \"0bbd3146-8a83-4cdd-9b41-410b53b540f4\") " pod="openshift-image-registry/node-ca-z856v" Jul 07 13:14:04.833416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833349 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-run-netns\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.833416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833367 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-var-lib-openvswitch\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.833416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833386 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28106265-9c28-4fe8-9678-40dc80bb00a8-host-slash\") pod \"iptables-alerter-cm9xz\" (UID: \"28106265-9c28-4fe8-9678-40dc80bb00a8\") " pod="openshift-network-operator/iptables-alerter-cm9xz" Jul 07 13:14:04.833416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833409 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-sysctl-conf\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.833784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833432 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-host\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.833784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833455 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29fc779c-15bd-423b-b3e9-30dbd743e902-tmp\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.833784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833467 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28106265-9c28-4fe8-9678-40dc80bb00a8-host-slash\") pod \"iptables-alerter-cm9xz\" (UID: \"28106265-9c28-4fe8-9678-40dc80bb00a8\") " pod="openshift-network-operator/iptables-alerter-cm9xz" Jul 07 13:14:04.833784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833479 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/58a47e74-7047-4c56-b452-5d41956fd31f-multus-daemon-config\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.833784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833501 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-run-systemd\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.833784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833528 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-run-ovn\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.833784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833554 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x4x6\" (UniqueName: \"kubernetes.io/projected/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-kube-api-access-4x4x6\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.833784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833580 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28106265-9c28-4fe8-9678-40dc80bb00a8-iptables-alerter-script\") pod \"iptables-alerter-cm9xz\" (UID: \"28106265-9c28-4fe8-9678-40dc80bb00a8\") " pod="openshift-network-operator/iptables-alerter-cm9xz" Jul 07 13:14:04.833784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833678 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-registration-dir\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.833784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833698 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkpwd\" (UniqueName: \"kubernetes.io/projected/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-kube-api-access-zkpwd\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.833784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833713 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klqn4\" (UniqueName: \"kubernetes.io/projected/29fc779c-15bd-423b-b3e9-30dbd743e902-kube-api-access-klqn4\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.833784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833729 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/331b85aa-4b8d-4efb-bcdb-4cec4bdae545-konnectivity-ca\") pod \"konnectivity-agent-hm7dm\" (UID: \"331b85aa-4b8d-4efb-bcdb-4cec4bdae545\") " pod="kube-system/konnectivity-agent-hm7dm" Jul 07 13:14:04.833784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833754 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-cni-bin\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.833784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833777 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.834224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833794 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-sys-fs\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.834224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833841 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-sysconfig\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.834224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833867 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-sys\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.834224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833894 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/58a47e74-7047-4c56-b452-5d41956fd31f-cni-binary-copy\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.834224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833918 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/331b85aa-4b8d-4efb-bcdb-4cec4bdae545-agent-certs\") pod \"konnectivity-agent-hm7dm\" (UID: \"331b85aa-4b8d-4efb-bcdb-4cec4bdae545\") " pod="kube-system/konnectivity-agent-hm7dm" Jul 07 13:14:04.834224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833936 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-cni-netd\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.834224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833954 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-cnibin\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.834224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833969 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0bbd3146-8a83-4cdd-9b41-410b53b540f4-serviceca\") pod \"node-ca-z856v\" (UID: \"0bbd3146-8a83-4cdd-9b41-410b53b540f4\") " pod="openshift-image-registry/node-ca-z856v" Jul 07 13:14:04.834224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.833974 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx7wt\" (UniqueName: \"kubernetes.io/projected/7416c196-b925-447d-8a35-a5c5421d15d1-kube-api-access-lx7wt\") pod \"node-resolver-q9ck9\" (UID: \"7416c196-b925-447d-8a35-a5c5421d15d1\") " pod="openshift-dns/node-resolver-q9ck9" Jul 07 13:14:04.834224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834014 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-cnibin\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.834224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834040 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-var-lib-cni-multus\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.834224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834082 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-system-cni-dir\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.834224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834104 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q78dq\" (UniqueName: \"kubernetes.io/projected/0bbd3146-8a83-4cdd-9b41-410b53b540f4-kube-api-access-q78dq\") pod \"node-ca-z856v\" (UID: \"0bbd3146-8a83-4cdd-9b41-410b53b540f4\") " pod="openshift-image-registry/node-ca-z856v" Jul 07 13:14:04.834224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834124 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28106265-9c28-4fe8-9678-40dc80bb00a8-iptables-alerter-script\") pod \"iptables-alerter-cm9xz\" (UID: \"28106265-9c28-4fe8-9678-40dc80bb00a8\") " pod="openshift-network-operator/iptables-alerter-cm9xz" Jul 07 13:14:04.834863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834156 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-system-cni-dir\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.834863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834124 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-run\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.834863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834193 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-multus-socket-dir-parent\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.834863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834220 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-var-lib-kubelet\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.834863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834260 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fbee5bee-337d-40af-baef-deee6657a83a-ovn-node-metrics-cert\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.834863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834295 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.834863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834321 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-modprobe-d\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.834863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834350 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-log-socket\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.834863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834375 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7416c196-b925-447d-8a35-a5c5421d15d1-tmp-dir\") pod \"node-resolver-q9ck9\" (UID: \"7416c196-b925-447d-8a35-a5c5421d15d1\") " pod="openshift-dns/node-resolver-q9ck9" Jul 07 13:14:04.834863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834401 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-etc-kubernetes\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.834863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834426 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.834863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834452 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.834863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834480 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7416c196-b925-447d-8a35-a5c5421d15d1-hosts-file\") pod \"node-resolver-q9ck9\" (UID: \"7416c196-b925-447d-8a35-a5c5421d15d1\") " pod="openshift-dns/node-resolver-q9ck9" Jul 07 13:14:04.834863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834507 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-socket-dir\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.835340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834562 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-sysctl-d\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.835340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834591 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-hostroot\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.835340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834598 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.835340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834618 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpt2m\" (UniqueName: \"kubernetes.io/projected/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-kube-api-access-jpt2m\") pod \"network-metrics-daemon-6l86h\" (UID: \"f63deb5a-7f0d-4cca-a94a-3f3e0892379b\") " pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:04.835340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834644 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-systemd-units\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.835340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834688 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-cni-binary-copy\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.835340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834713 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-os-release\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.835340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834740 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.835340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834770 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-kubelet\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.835340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834795 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-slash\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.835340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834819 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fbee5bee-337d-40af-baef-deee6657a83a-ovnkube-script-lib\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.835340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834841 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.835340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834845 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:04.835340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834893 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srktn\" (UniqueName: \"kubernetes.io/projected/28106265-9c28-4fe8-9678-40dc80bb00a8-kube-api-access-srktn\") pod \"iptables-alerter-cm9xz\" (UID: \"28106265-9c28-4fe8-9678-40dc80bb00a8\") " pod="openshift-network-operator/iptables-alerter-cm9xz" Jul 07 13:14:04.835758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834923 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-etc-selinux\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.835758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834950 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-systemd\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.835758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834976 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-cnibin\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.835758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.834992 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.835758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.835007 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l8lx\" (UniqueName: \"kubernetes.io/projected/58a47e74-7047-4c56-b452-5d41956fd31f-kube-api-access-7l8lx\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.835758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.835032 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-run-openvswitch\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.835758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.835068 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.835758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.835090 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-device-dir\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.835758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.835105 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-lib-modules\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.835758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.835120 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-var-lib-cni-bin\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.835758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.835137 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-cni-binary-copy\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.835758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.835146 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-os-release\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.835758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.835183 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-kubernetes\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.835758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.835199 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-os-release\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.842101 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.842079 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tjk58" Jul 07 13:14:04.845681 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.845644 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 13:14:04.849202 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.849178 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x4x6\" (UniqueName: \"kubernetes.io/projected/ab6cbacd-df97-40ea-9abd-e52a4f49af4d-kube-api-access-4x4x6\") pod \"multus-additional-cni-plugins-h8rgd\" (UID: \"ab6cbacd-df97-40ea-9abd-e52a4f49af4d\") " pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:04.849289 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.849202 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srktn\" (UniqueName: \"kubernetes.io/projected/28106265-9c28-4fe8-9678-40dc80bb00a8-kube-api-access-srktn\") pod \"iptables-alerter-cm9xz\" (UID: \"28106265-9c28-4fe8-9678-40dc80bb00a8\") " pod="openshift-network-operator/iptables-alerter-cm9xz" Jul 07 13:14:04.849289 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.849253 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q78dq\" (UniqueName: \"kubernetes.io/projected/0bbd3146-8a83-4cdd-9b41-410b53b540f4-kube-api-access-q78dq\") pod \"node-ca-z856v\" (UID: \"0bbd3146-8a83-4cdd-9b41-410b53b540f4\") " pod="openshift-image-registry/node-ca-z856v" Jul 07 13:14:04.935470 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935430 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-var-lib-kubelet\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.935470 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935478 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fbee5bee-337d-40af-baef-deee6657a83a-ovn-node-metrics-cert\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.935740 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935504 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-modprobe-d\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.935740 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935525 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-log-socket\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.935740 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935546 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7416c196-b925-447d-8a35-a5c5421d15d1-tmp-dir\") pod \"node-resolver-q9ck9\" (UID: \"7416c196-b925-447d-8a35-a5c5421d15d1\") " pod="openshift-dns/node-resolver-q9ck9" Jul 07 13:14:04.935740 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935565 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-etc-kubernetes\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.935740 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935617 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7416c196-b925-447d-8a35-a5c5421d15d1-hosts-file\") pod \"node-resolver-q9ck9\" (UID: \"7416c196-b925-447d-8a35-a5c5421d15d1\") " pod="openshift-dns/node-resolver-q9ck9" Jul 07 13:14:04.935740 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935658 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-socket-dir\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.935740 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935682 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-sysctl-d\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.935740 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935705 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-hostroot\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.935740 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935739 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpt2m\" (UniqueName: \"kubernetes.io/projected/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-kube-api-access-jpt2m\") pod \"network-metrics-daemon-6l86h\" (UID: \"f63deb5a-7f0d-4cca-a94a-3f3e0892379b\") " pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:04.936188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935764 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-systemd-units\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.936188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935794 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-os-release\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.936188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935820 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.936188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935846 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-kubelet\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.936188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935871 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-slash\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.936188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935912 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fbee5bee-337d-40af-baef-deee6657a83a-ovnkube-script-lib\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.936188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935942 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:04.936188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935969 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-etc-selinux\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.936188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.935993 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-systemd\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.936188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936018 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-cnibin\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.936188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936045 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l8lx\" (UniqueName: \"kubernetes.io/projected/58a47e74-7047-4c56-b452-5d41956fd31f-kube-api-access-7l8lx\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.936188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936074 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-hostroot\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.936188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936101 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-run-openvswitch\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.936188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936135 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936163 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-device-dir\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936172 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7416c196-b925-447d-8a35-a5c5421d15d1-hosts-file\") pod \"node-resolver-q9ck9\" (UID: \"7416c196-b925-447d-8a35-a5c5421d15d1\") " pod="openshift-dns/node-resolver-q9ck9" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936189 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-lib-modules\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936219 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-var-lib-cni-bin\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936247 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-kubernetes\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936284 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-tuned\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936293 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-socket-dir\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936336 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-run-netns\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936362 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-multus-conf-dir\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936387 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-etc-openvswitch\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936414 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-var-lib-kubelet\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936433 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-sysctl-d\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936441 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-system-cni-dir\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936457 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7416c196-b925-447d-8a35-a5c5421d15d1-tmp-dir\") pod \"node-resolver-q9ck9\" (UID: \"7416c196-b925-447d-8a35-a5c5421d15d1\") " pod="openshift-dns/node-resolver-q9ck9" Jul 07 13:14:04.936947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936484 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-run-k8s-cni-cncf-io\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936515 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-run-k8s-cni-cncf-io\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936535 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs\") pod \"network-metrics-daemon-6l86h\" (UID: \"f63deb5a-7f0d-4cca-a94a-3f3e0892379b\") " pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936550 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-system-cni-dir\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936568 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-node-log\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936595 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fbee5bee-337d-40af-baef-deee6657a83a-ovnkube-config\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936613 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfh7m\" (UniqueName: \"kubernetes.io/projected/fbee5bee-337d-40af-baef-deee6657a83a-kube-api-access-gfh7m\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936613 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-modprobe-d\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936635 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/730e3c49-83b7-4a1d-9b7c-d5621955f6df-kubelet-config\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936674 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-var-lib-kubelet\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936682 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-multus-cni-dir\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936712 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-run-multus-certs\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936729 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fbee5bee-337d-40af-baef-deee6657a83a-env-overrides\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936775 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-systemd-units\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936813 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/730e3c49-83b7-4a1d-9b7c-d5621955f6df-dbus\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:04.937726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936848 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zxrv\" (UniqueName: \"kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv\") pod \"network-check-target-7cw9x\" (UID: \"6037cecb-d996-4bd8-a3d3-733a764f5163\") " pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:04.938609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936880 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-run-netns\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.938609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936908 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-var-lib-openvswitch\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.938609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.936958 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-sysctl-conf\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.938609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937020 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-host\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.938609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937050 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29fc779c-15bd-423b-b3e9-30dbd743e902-tmp\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.938609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937077 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/58a47e74-7047-4c56-b452-5d41956fd31f-multus-daemon-config\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.938609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937127 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-run-systemd\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.938609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937153 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-run-ovn\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.938609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937185 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-registration-dir\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.938609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937213 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkpwd\" (UniqueName: \"kubernetes.io/projected/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-kube-api-access-zkpwd\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.938609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937241 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fbee5bee-337d-40af-baef-deee6657a83a-env-overrides\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.938609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937243 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klqn4\" (UniqueName: \"kubernetes.io/projected/29fc779c-15bd-423b-b3e9-30dbd743e902-kube-api-access-klqn4\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.938609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937315 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/331b85aa-4b8d-4efb-bcdb-4cec4bdae545-konnectivity-ca\") pod \"konnectivity-agent-hm7dm\" (UID: \"331b85aa-4b8d-4efb-bcdb-4cec4bdae545\") " pod="kube-system/konnectivity-agent-hm7dm" Jul 07 13:14:04.938609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937371 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-cni-bin\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937410 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937455 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-sys-fs\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937484 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-sysconfig\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937526 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-sys\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937576 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-os-release\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937622 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/58a47e74-7047-4c56-b452-5d41956fd31f-cni-binary-copy\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937667 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/331b85aa-4b8d-4efb-bcdb-4cec4bdae545-agent-certs\") pod \"konnectivity-agent-hm7dm\" (UID: \"331b85aa-4b8d-4efb-bcdb-4cec4bdae545\") " pod="kube-system/konnectivity-agent-hm7dm" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937667 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937706 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-kubelet\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937739 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-cni-netd\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937741 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-log-socket\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937777 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx7wt\" (UniqueName: \"kubernetes.io/projected/7416c196-b925-447d-8a35-a5c5421d15d1-kube-api-access-lx7wt\") pod \"node-resolver-q9ck9\" (UID: \"7416c196-b925-447d-8a35-a5c5421d15d1\") " pod="openshift-dns/node-resolver-q9ck9" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937822 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-var-lib-cni-multus\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937853 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-run\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.939322 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:04.937864 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:14:04.940121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-multus-socket-dir-parent\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.940121 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:04.937953 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs podName:f63deb5a-7f0d-4cca-a94a-3f3e0892379b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:05.437922062 +0000 UTC m=+3.116483376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs") pod "network-metrics-daemon-6l86h" (UID: "f63deb5a-7f0d-4cca-a94a-3f3e0892379b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:14:04.940121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.937998 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-node-log\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.940121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.938002 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-multus-socket-dir-parent\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.940121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.938607 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fbee5bee-337d-40af-baef-deee6657a83a-ovnkube-config\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.940121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.938672 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fbee5bee-337d-40af-baef-deee6657a83a-ovnkube-script-lib\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.940121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.938695 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-slash\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.940121 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:04.938781 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:14:04.940121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.938785 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/730e3c49-83b7-4a1d-9b7c-d5621955f6df-kubelet-config\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:04.940121 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:04.938887 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret podName:730e3c49-83b7-4a1d-9b7c-d5621955f6df nodeName:}" failed. No retries permitted until 2026-07-07 13:14:05.438870426 +0000 UTC m=+3.117431744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret") pod "global-pull-secret-syncer-j2rqx" (UID: "730e3c49-83b7-4a1d-9b7c-d5621955f6df") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:14:04.940121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.938958 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-multus-cni-dir\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.940121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939004 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-etc-selinux\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.940121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939020 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-run-multus-certs\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.940121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939082 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-systemd\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939142 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-cnibin\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939309 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-run-openvswitch\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939380 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-etc-kubernetes\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939440 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-cni-bin\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939499 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-lib-modules\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939509 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939570 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-sysconfig\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939606 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/331b85aa-4b8d-4efb-bcdb-4cec4bdae545-konnectivity-ca\") pod \"konnectivity-agent-hm7dm\" (UID: \"331b85aa-4b8d-4efb-bcdb-4cec4bdae545\") " pod="kube-system/konnectivity-agent-hm7dm" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939626 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-sys-fs\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939674 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-kubernetes\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939689 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-sys\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939376 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.939572 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-device-dir\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.940162 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/58a47e74-7047-4c56-b452-5d41956fd31f-cni-binary-copy\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.941621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.940241 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-var-lib-cni-bin\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.940591 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-run-systemd\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.940593 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-etc-openvswitch\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.940690 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-multus-conf-dir\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.940707 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-var-lib-kubelet\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.940737 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-var-lib-cni-multus\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.940787 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-run\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.940810 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/730e3c49-83b7-4a1d-9b7c-d5621955f6df-dbus\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.940851 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58a47e74-7047-4c56-b452-5d41956fd31f-host-run-netns\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.940909 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-cni-netd\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.940961 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-var-lib-openvswitch\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.941018 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-sysctl-conf\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.941031 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29fc779c-15bd-423b-b3e9-30dbd743e902-host\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.941082 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-registration-dir\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.941089 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-run-ovn\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.942421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.941137 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbee5bee-337d-40af-baef-deee6657a83a-host-run-netns\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.943221 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.941160 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fbee5bee-337d-40af-baef-deee6657a83a-ovn-node-metrics-cert\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.943221 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.941311 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/58a47e74-7047-4c56-b452-5d41956fd31f-multus-daemon-config\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.943221 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.942686 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29fc779c-15bd-423b-b3e9-30dbd743e902-tmp\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.943221 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.943093 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29fc779c-15bd-423b-b3e9-30dbd743e902-etc-tuned\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.945171 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.945143 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/331b85aa-4b8d-4efb-bcdb-4cec4bdae545-agent-certs\") pod \"konnectivity-agent-hm7dm\" (UID: \"331b85aa-4b8d-4efb-bcdb-4cec4bdae545\") " pod="kube-system/konnectivity-agent-hm7dm" Jul 07 13:14:04.950417 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.950352 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l8lx\" (UniqueName: \"kubernetes.io/projected/58a47e74-7047-4c56-b452-5d41956fd31f-kube-api-access-7l8lx\") pod \"multus-twksx\" (UID: \"58a47e74-7047-4c56-b452-5d41956fd31f\") " pod="openshift-multus/multus-twksx" Jul 07 13:14:04.950417 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.950354 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx7wt\" (UniqueName: \"kubernetes.io/projected/7416c196-b925-447d-8a35-a5c5421d15d1-kube-api-access-lx7wt\") pod \"node-resolver-q9ck9\" (UID: \"7416c196-b925-447d-8a35-a5c5421d15d1\") " pod="openshift-dns/node-resolver-q9ck9" Jul 07 13:14:04.951933 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.951904 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpt2m\" (UniqueName: \"kubernetes.io/projected/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-kube-api-access-jpt2m\") pod \"network-metrics-daemon-6l86h\" (UID: \"f63deb5a-7f0d-4cca-a94a-3f3e0892379b\") " pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:04.952270 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.952253 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klqn4\" (UniqueName: \"kubernetes.io/projected/29fc779c-15bd-423b-b3e9-30dbd743e902-kube-api-access-klqn4\") pod \"tuned-4gqpp\" (UID: \"29fc779c-15bd-423b-b3e9-30dbd743e902\") " pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:04.957921 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:04.957666 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:14:04.957921 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:04.957690 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:14:04.957921 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:04.957705 2568 projected.go:196] Error preparing data for projected volume kube-api-access-9zxrv for pod openshift-network-diagnostics/network-check-target-7cw9x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:04.957921 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:04.957761 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv podName:6037cecb-d996-4bd8-a3d3-733a764f5163 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:05.457743349 +0000 UTC m=+3.136304674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9zxrv" (UniqueName: "kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv") pod "network-check-target-7cw9x" (UID: "6037cecb-d996-4bd8-a3d3-733a764f5163") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:04.960167 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.960123 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkpwd\" (UniqueName: \"kubernetes.io/projected/54e31e35-d1da-4cf9-a0be-beda43aeb2bb-kube-api-access-zkpwd\") pod \"aws-ebs-csi-driver-node-4qd87\" (UID: \"54e31e35-d1da-4cf9-a0be-beda43aeb2bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:04.960787 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.960766 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfh7m\" (UniqueName: \"kubernetes.io/projected/fbee5bee-337d-40af-baef-deee6657a83a-kube-api-access-gfh7m\") pod \"ovnkube-node-8wjmr\" (UID: \"fbee5bee-337d-40af-baef-deee6657a83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:04.971545 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.971387 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" event={"ID":"c861ca34efcd36b7887620391a65da0e","Type":"ContainerStarted","Data":"9a5574c4ea7e2b8eec0f1e83b1b986b1d5031e032600f396cea288214b60e8ac"} Jul 07 13:14:04.973005 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:04.972978 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" event={"ID":"a1ee8660983e09b5aeea7a0e782f81de","Type":"ContainerStarted","Data":"c36462fb48f4b9f50facf9f7c0ba382b6a7dc97842441fab23bf31417679feaa"} Jul 07 13:14:05.026272 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.026238 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h8rgd" Jul 07 13:14:05.031109 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.031081 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cm9xz" Jul 07 13:14:05.039245 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.039143 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z856v" Jul 07 13:14:05.039939 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:05.039911 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6cbacd_df97_40ea_9abd_e52a4f49af4d.slice/crio-22482ff2139cd0ab6a643981db8d3c56ede26b1da76043386923cde8cd4c2dd5 WatchSource:0}: Error finding container 22482ff2139cd0ab6a643981db8d3c56ede26b1da76043386923cde8cd4c2dd5: Status 404 returned error can't find the container with id 22482ff2139cd0ab6a643981db8d3c56ede26b1da76043386923cde8cd4c2dd5 Jul 07 13:14:05.044118 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.043961 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hm7dm" Jul 07 13:14:05.049864 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.049708 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:05.053062 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:05.053030 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bbd3146_8a83_4cdd_9b41_410b53b540f4.slice/crio-075f55d5e10430af21795d554e85ea496f8cd64fcc0d4aacdbb0bf9fdaa2d894 WatchSource:0}: Error finding container 075f55d5e10430af21795d554e85ea496f8cd64fcc0d4aacdbb0bf9fdaa2d894: Status 404 returned error can't find the container with id 075f55d5e10430af21795d554e85ea496f8cd64fcc0d4aacdbb0bf9fdaa2d894 Jul 07 13:14:05.056737 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.056709 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q9ck9" Jul 07 13:14:05.062353 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:05.062331 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod331b85aa_4b8d_4efb_bcdb_4cec4bdae545.slice/crio-58622e732f4c679dba0555afe7de333e871521bdde4b841efad7c912a7415a8a WatchSource:0}: Error finding container 58622e732f4c679dba0555afe7de333e871521bdde4b841efad7c912a7415a8a: Status 404 returned error can't find the container with id 58622e732f4c679dba0555afe7de333e871521bdde4b841efad7c912a7415a8a Jul 07 13:14:05.066789 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.066728 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" Jul 07 13:14:05.072366 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.072233 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" Jul 07 13:14:05.073805 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:05.073779 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7416c196_b925_447d_8a35_a5c5421d15d1.slice/crio-bac3bc8846a7a5567028c8b229d526fa2e4f5933cfae8bff27af5cf0e7cb93de WatchSource:0}: Error finding container bac3bc8846a7a5567028c8b229d526fa2e4f5933cfae8bff27af5cf0e7cb93de: Status 404 returned error can't find the container with id bac3bc8846a7a5567028c8b229d526fa2e4f5933cfae8bff27af5cf0e7cb93de Jul 07 13:14:05.076630 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.076582 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-twksx" Jul 07 13:14:05.092762 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:05.092731 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29fc779c_15bd_423b_b3e9_30dbd743e902.slice/crio-fd6001329bcfc5a8792840858c88ef1b836dd280e149ab44d912667bfb7f3735 WatchSource:0}: Error finding container fd6001329bcfc5a8792840858c88ef1b836dd280e149ab44d912667bfb7f3735: Status 404 returned error can't find the container with id fd6001329bcfc5a8792840858c88ef1b836dd280e149ab44d912667bfb7f3735 Jul 07 13:14:05.093512 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:05.093491 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a47e74_7047_4c56_b452_5d41956fd31f.slice/crio-f5689284832529ab00e9f0f0d0ab3e3f7555952a06ffe26345ba7a03bef46b29 WatchSource:0}: Error finding container f5689284832529ab00e9f0f0d0ab3e3f7555952a06ffe26345ba7a03bef46b29: Status 404 returned error can't find the container with id f5689284832529ab00e9f0f0d0ab3e3f7555952a06ffe26345ba7a03bef46b29 Jul 07 13:14:05.219225 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.219123 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:14:05.442494 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.442445 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:05.442707 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.442511 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs\") pod \"network-metrics-daemon-6l86h\" (UID: \"f63deb5a-7f0d-4cca-a94a-3f3e0892379b\") " pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:05.442707 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:05.442681 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:14:05.442707 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:05.442685 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:14:05.442879 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:05.442740 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs podName:f63deb5a-7f0d-4cca-a94a-3f3e0892379b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:06.442725946 +0000 UTC m=+4.121287270 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs") pod "network-metrics-daemon-6l86h" (UID: "f63deb5a-7f0d-4cca-a94a-3f3e0892379b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:14:05.442879 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:05.442754 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret podName:730e3c49-83b7-4a1d-9b7c-d5621955f6df nodeName:}" failed. No retries permitted until 2026-07-07 13:14:06.442748076 +0000 UTC m=+4.121309387 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret") pod "global-pull-secret-syncer-j2rqx" (UID: "730e3c49-83b7-4a1d-9b7c-d5621955f6df") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:14:05.543996 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.543901 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zxrv\" (UniqueName: \"kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv\") pod \"network-check-target-7cw9x\" (UID: \"6037cecb-d996-4bd8-a3d3-733a764f5163\") " pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:05.544164 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:05.544115 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:14:05.544164 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:05.544145 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:14:05.544164 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:05.544160 2568 projected.go:196] Error preparing data for projected volume kube-api-access-9zxrv for pod openshift-network-diagnostics/network-check-target-7cw9x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:05.544327 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:05.544231 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv podName:6037cecb-d996-4bd8-a3d3-733a764f5163 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:06.544209974 +0000 UTC m=+4.222771301 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9zxrv" (UniqueName: "kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv") pod "network-check-target-7cw9x" (UID: "6037cecb-d996-4bd8-a3d3-733a764f5163") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:05.844751 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.844648 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 13:09:05 +0000 UTC" deadline="2028-03-21 15:03:26.603380188 +0000 UTC" Jul 07 13:14:05.844751 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.844705 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14953h49m20.758681033s" Jul 07 13:14:05.965124 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.965088 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:05.965337 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.965093 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:05.965337 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.965224 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:05.965337 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:05.965218 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:05.965662 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:05.965340 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:05.965662 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:05.965443 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:05.977128 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.977087 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" event={"ID":"fbee5bee-337d-40af-baef-deee6657a83a","Type":"ContainerStarted","Data":"92ed2bba5d4c7410ea560cf883a3e520135a6176e11447f927065bde6ffea1b8"} Jul 07 13:14:05.980398 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.980334 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hm7dm" event={"ID":"331b85aa-4b8d-4efb-bcdb-4cec4bdae545","Type":"ContainerStarted","Data":"58622e732f4c679dba0555afe7de333e871521bdde4b841efad7c912a7415a8a"} Jul 07 13:14:05.982214 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.982174 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cm9xz" event={"ID":"28106265-9c28-4fe8-9678-40dc80bb00a8","Type":"ContainerStarted","Data":"e64d83ca84ab034d8243237a2c6173e105e8cbc1cc456e56d8f34e3965e8ed17"} Jul 07 13:14:05.984184 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.984096 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h8rgd" event={"ID":"ab6cbacd-df97-40ea-9abd-e52a4f49af4d","Type":"ContainerStarted","Data":"22482ff2139cd0ab6a643981db8d3c56ede26b1da76043386923cde8cd4c2dd5"} Jul 07 13:14:05.989261 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.989127 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" event={"ID":"54e31e35-d1da-4cf9-a0be-beda43aeb2bb","Type":"ContainerStarted","Data":"3b936dd3083ade2bb9637c869000f1bf95de97d5ddb9c36a99c7ef1a8fd1ffc0"} Jul 07 13:14:05.991137 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.991109 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z856v" event={"ID":"0bbd3146-8a83-4cdd-9b41-410b53b540f4","Type":"ContainerStarted","Data":"075f55d5e10430af21795d554e85ea496f8cd64fcc0d4aacdbb0bf9fdaa2d894"} Jul 07 13:14:05.993735 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.993701 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-twksx" event={"ID":"58a47e74-7047-4c56-b452-5d41956fd31f","Type":"ContainerStarted","Data":"f5689284832529ab00e9f0f0d0ab3e3f7555952a06ffe26345ba7a03bef46b29"} Jul 07 13:14:05.996241 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.996209 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" event={"ID":"29fc779c-15bd-423b-b3e9-30dbd743e902","Type":"ContainerStarted","Data":"fd6001329bcfc5a8792840858c88ef1b836dd280e149ab44d912667bfb7f3735"} Jul 07 13:14:05.998035 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:05.998008 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q9ck9" event={"ID":"7416c196-b925-447d-8a35-a5c5421d15d1","Type":"ContainerStarted","Data":"bac3bc8846a7a5567028c8b229d526fa2e4f5933cfae8bff27af5cf0e7cb93de"} Jul 07 13:14:06.453804 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:06.453761 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:06.453995 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:06.453841 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs\") pod \"network-metrics-daemon-6l86h\" (UID: \"f63deb5a-7f0d-4cca-a94a-3f3e0892379b\") " pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:06.454571 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:06.454070 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:14:06.454571 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:06.454155 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs podName:f63deb5a-7f0d-4cca-a94a-3f3e0892379b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:08.45412429 +0000 UTC m=+6.132685605 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs") pod "network-metrics-daemon-6l86h" (UID: "f63deb5a-7f0d-4cca-a94a-3f3e0892379b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:14:06.454571 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:06.454175 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:14:06.454571 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:06.454243 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret podName:730e3c49-83b7-4a1d-9b7c-d5621955f6df nodeName:}" failed. No retries permitted until 2026-07-07 13:14:08.454225023 +0000 UTC m=+6.132786350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret") pod "global-pull-secret-syncer-j2rqx" (UID: "730e3c49-83b7-4a1d-9b7c-d5621955f6df") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:14:06.554478 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:06.554432 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zxrv\" (UniqueName: \"kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv\") pod \"network-check-target-7cw9x\" (UID: \"6037cecb-d996-4bd8-a3d3-733a764f5163\") " pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:06.554810 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:06.554666 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:14:06.554810 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:06.554697 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:14:06.554810 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:06.554711 2568 projected.go:196] Error preparing data for projected volume kube-api-access-9zxrv for pod openshift-network-diagnostics/network-check-target-7cw9x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:06.554810 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:06.554774 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv podName:6037cecb-d996-4bd8-a3d3-733a764f5163 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:08.55475442 +0000 UTC m=+6.233315756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9zxrv" (UniqueName: "kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv") pod "network-check-target-7cw9x" (UID: "6037cecb-d996-4bd8-a3d3-733a764f5163") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:06.845919 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:06.845787 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 13:09:05 +0000 UTC" deadline="2028-03-13 13:45:54.178719241 +0000 UTC" Jul 07 13:14:06.845919 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:06.845830 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14760h31m47.332894096s" Jul 07 13:14:07.966094 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:07.965197 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:07.966094 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:07.965359 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:07.966094 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:07.965814 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:07.966094 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:07.965922 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:07.966094 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:07.965983 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:07.966094 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:07.966054 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:08.472302 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:08.472241 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:08.472511 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:08.472357 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs\") pod \"network-metrics-daemon-6l86h\" (UID: \"f63deb5a-7f0d-4cca-a94a-3f3e0892379b\") " pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:08.472511 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:08.472469 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:14:08.472645 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:08.472539 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:14:08.472645 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:08.472547 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret podName:730e3c49-83b7-4a1d-9b7c-d5621955f6df nodeName:}" failed. No retries permitted until 2026-07-07 13:14:12.472525 +0000 UTC m=+10.151086348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret") pod "global-pull-secret-syncer-j2rqx" (UID: "730e3c49-83b7-4a1d-9b7c-d5621955f6df") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:14:08.472645 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:08.472642 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs podName:f63deb5a-7f0d-4cca-a94a-3f3e0892379b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:12.472607228 +0000 UTC m=+10.151168548 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs") pod "network-metrics-daemon-6l86h" (UID: "f63deb5a-7f0d-4cca-a94a-3f3e0892379b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:14:08.573105 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:08.573035 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zxrv\" (UniqueName: \"kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv\") pod \"network-check-target-7cw9x\" (UID: \"6037cecb-d996-4bd8-a3d3-733a764f5163\") " pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:08.573308 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:08.573209 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:14:08.573308 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:08.573233 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:14:08.573308 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:08.573247 2568 projected.go:196] Error preparing data for projected volume kube-api-access-9zxrv for pod openshift-network-diagnostics/network-check-target-7cw9x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:08.573474 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:08.573319 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv podName:6037cecb-d996-4bd8-a3d3-733a764f5163 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:12.573299184 +0000 UTC m=+10.251860499 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9zxrv" (UniqueName: "kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv") pod "network-check-target-7cw9x" (UID: "6037cecb-d996-4bd8-a3d3-733a764f5163") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:09.965603 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:09.965533 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:09.965603 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:09.965568 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:09.965603 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:09.965568 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:09.966148 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:09.965700 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:09.966148 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:09.965830 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:09.966148 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:09.965911 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:11.965612 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:11.965560 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:11.965612 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:11.965604 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:11.966110 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:11.965685 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:11.966110 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:11.965701 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:11.966110 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:11.965796 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:11.966110 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:11.965863 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:12.500994 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:12.500956 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:12.500994 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:12.501014 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs\") pod \"network-metrics-daemon-6l86h\" (UID: \"f63deb5a-7f0d-4cca-a94a-3f3e0892379b\") " pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:12.501228 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:12.501122 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:14:12.501228 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:12.501131 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:14:12.501228 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:12.501191 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs podName:f63deb5a-7f0d-4cca-a94a-3f3e0892379b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:20.501175012 +0000 UTC m=+18.179736343 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs") pod "network-metrics-daemon-6l86h" (UID: "f63deb5a-7f0d-4cca-a94a-3f3e0892379b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:14:12.501228 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:12.501207 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret podName:730e3c49-83b7-4a1d-9b7c-d5621955f6df nodeName:}" failed. No retries permitted until 2026-07-07 13:14:20.501200832 +0000 UTC m=+18.179762141 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret") pod "global-pull-secret-syncer-j2rqx" (UID: "730e3c49-83b7-4a1d-9b7c-d5621955f6df") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:14:12.602243 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:12.602208 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zxrv\" (UniqueName: \"kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv\") pod \"network-check-target-7cw9x\" (UID: \"6037cecb-d996-4bd8-a3d3-733a764f5163\") " pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:12.602441 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:12.602419 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:14:12.602505 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:12.602446 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:14:12.602505 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:12.602457 2568 projected.go:196] Error preparing data for projected volume kube-api-access-9zxrv for pod openshift-network-diagnostics/network-check-target-7cw9x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:12.602626 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:12.602513 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv podName:6037cecb-d996-4bd8-a3d3-733a764f5163 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:20.602495296 +0000 UTC m=+18.281056626 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9zxrv" (UniqueName: "kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv") pod "network-check-target-7cw9x" (UID: "6037cecb-d996-4bd8-a3d3-733a764f5163") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:13.965139 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:13.964882 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:13.965603 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:13.964911 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:13.965603 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:13.965254 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:13.965603 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:13.965369 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:13.965603 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:13.964932 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:13.965603 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:13.965476 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:15.965512 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:15.965263 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:15.965928 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:15.965295 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:15.965928 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:15.965296 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:15.965928 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:15.965705 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:15.965928 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:15.965891 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:15.966072 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:15.965962 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:16.026314 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.026277 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" event={"ID":"a1ee8660983e09b5aeea7a0e782f81de","Type":"ContainerStarted","Data":"9bf25b072982aa27733ec761be587aa2bf1858a403443e0d8ea63ef5f84a52ba"} Jul 07 13:14:16.028090 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.028063 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" event={"ID":"54e31e35-d1da-4cf9-a0be-beda43aeb2bb","Type":"ContainerStarted","Data":"0d33fc03ca865bfc0502cab05b4b78c22f73a0d488332773704b44790db6ae8d"} Jul 07 13:14:16.029638 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.029610 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z856v" event={"ID":"0bbd3146-8a83-4cdd-9b41-410b53b540f4","Type":"ContainerStarted","Data":"7c76c84ec8f910c4bcc33ba6c113401b98cfe640ebb0813e7c7a92157d78e61c"} Jul 07 13:14:16.031287 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.031259 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" event={"ID":"29fc779c-15bd-423b-b3e9-30dbd743e902","Type":"ContainerStarted","Data":"30ae67f8f9bbc6c6d8e5682035da337c4c73d6146dae5e4b915d3638313c57f4"} Jul 07 13:14:16.032888 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.032851 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q9ck9" event={"ID":"7416c196-b925-447d-8a35-a5c5421d15d1","Type":"ContainerStarted","Data":"2d952bd544869460e35b6610b53fbd5011ec5f173eed61534c663e73ed76b45f"} Jul 07 13:14:16.034392 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.034364 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hm7dm" event={"ID":"331b85aa-4b8d-4efb-bcdb-4cec4bdae545","Type":"ContainerStarted","Data":"3fddbeff67a757334510e8adcdd8706837f8008ac412dc8773706ef8928ec425"} Jul 07 13:14:16.035954 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.035931 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h8rgd" event={"ID":"ab6cbacd-df97-40ea-9abd-e52a4f49af4d","Type":"ContainerStarted","Data":"0934525cb6e292cf739b3ab4d787c95fb36883a39ee549ffd383d12ee94f9eef"} Jul 07 13:14:16.037253 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.037234 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" event={"ID":"c861ca34efcd36b7887620391a65da0e","Type":"ContainerStarted","Data":"ad0b52ad60808e4317f816650ba31600ba80c13aeaafd8fd96aedf7df4a984c6"} Jul 07 13:14:16.096307 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.096201 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4gqpp" podStartSLOduration=2.7624771470000002 podStartE2EDuration="13.09618469s" totalImagesPullingTime="10.333707543s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:16.095513477 +0000 UTC m=+13.774074808" watchObservedRunningTime="2026-07-07 13:14:16.09618469 +0000 UTC m=+13.774746093" Jul 07 13:14:16.112394 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.112312 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-z856v" podStartSLOduration=2.762464211 podStartE2EDuration="13.112294954s" totalImagesPullingTime="10.349830743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:16.111829631 +0000 UTC m=+13.790390963" watchObservedRunningTime="2026-07-07 13:14:16.112294954 +0000 UTC m=+13.790856286" Jul 07 13:14:16.148230 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.148173 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hm7dm" podStartSLOduration=2.916059701 podStartE2EDuration="13.148155283s" totalImagesPullingTime="10.232095582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:16.128791458 +0000 UTC m=+13.807352793" watchObservedRunningTime="2026-07-07 13:14:16.148155283 +0000 UTC m=+13.826716614" Jul 07 13:14:16.148738 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.148699 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-q9ck9" podStartSLOduration=2.820849657 podStartE2EDuration="13.148686263s" totalImagesPullingTime="10.327836606s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:16.147619174 +0000 UTC m=+13.826180515" watchObservedRunningTime="2026-07-07 13:14:16.148686263 +0000 UTC m=+13.827247595" Jul 07 13:14:16.167017 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.166966 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" podStartSLOduration=12.166945066 podStartE2EDuration="12.166945066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:14:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:16.166430263 +0000 UTC m=+13.844991595" watchObservedRunningTime="2026-07-07 13:14:16.166945066 +0000 UTC m=+13.845506399" Jul 07 13:14:16.205743 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.205704 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:16.287524 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:16.287491 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:17.965166 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:17.965130 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:17.965754 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:17.965169 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:17.965754 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:17.965249 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:17.965754 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:17.965131 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:17.965754 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:17.965374 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:17.965754 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:17.965511 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:18.043502 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:18.043449 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cm9xz" event={"ID":"28106265-9c28-4fe8-9678-40dc80bb00a8","Type":"ContainerStarted","Data":"6024b8f5601123adf5e03336f39857912fef89f415aa24f29272406a02aa832f"} Jul 07 13:14:18.080368 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:18.080303 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cm9xz" podStartSLOduration=4.723655233 podStartE2EDuration="15.080280133s" totalImagesPullingTime="10.3566249s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:18.079438742 +0000 UTC m=+15.758000099" watchObservedRunningTime="2026-07-07 13:14:18.080280133 +0000 UTC m=+15.758841465" Jul 07 13:14:18.294555 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:18.294469 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:18.417430 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:18.417394 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:19.937773 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:19.937604 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:19.965444 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:19.965423 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:19.965553 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:19.965423 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:19.965553 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:19.965530 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:19.965688 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:19.965626 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:19.965688 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:19.965423 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:19.965775 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:19.965734 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:20.037141 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:20.037108 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:20.049263 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:20.049231 2568 generic.go:356] "Generic (PLEG): container finished" podID="ab6cbacd-df97-40ea-9abd-e52a4f49af4d" containerID="0934525cb6e292cf739b3ab4d787c95fb36883a39ee549ffd383d12ee94f9eef" exitCode=0 Jul 07 13:14:20.049444 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:20.049309 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h8rgd" event={"ID":"ab6cbacd-df97-40ea-9abd-e52a4f49af4d","Type":"ContainerDied","Data":"0934525cb6e292cf739b3ab4d787c95fb36883a39ee549ffd383d12ee94f9eef"} Jul 07 13:14:20.051023 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:20.051001 2568 generic.go:356] "Generic (PLEG): container finished" podID="a1ee8660983e09b5aeea7a0e782f81de" containerID="9bf25b072982aa27733ec761be587aa2bf1858a403443e0d8ea63ef5f84a52ba" exitCode=0 Jul 07 13:14:20.051132 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:20.051073 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" event={"ID":"a1ee8660983e09b5aeea7a0e782f81de","Type":"ContainerDied","Data":"9bf25b072982aa27733ec761be587aa2bf1858a403443e0d8ea63ef5f84a52ba"} Jul 07 13:14:20.053197 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:20.053144 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-twksx" event={"ID":"58a47e74-7047-4c56-b452-5d41956fd31f","Type":"ContainerStarted","Data":"ad5569d03d753d7184825c8909aa00852f63ae694f02ad16ec49f2890ff08350"} Jul 07 13:14:20.104018 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:20.103615 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-twksx" podStartSLOduration=2.321322489 podStartE2EDuration="17.1035952s" totalImagesPullingTime="14.782272711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:20.102856752 +0000 UTC m=+17.781418082" watchObservedRunningTime="2026-07-07 13:14:20.1035952 +0000 UTC m=+17.782156524" Jul 07 13:14:20.189780 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:20.189719 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hm7dm" Jul 07 13:14:20.190755 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:20.190734 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hm7dm" Jul 07 13:14:20.215041 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:20.214961 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:20.300222 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:20.300187 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:20.567742 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:20.567648 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs\") pod \"network-metrics-daemon-6l86h\" (UID: \"f63deb5a-7f0d-4cca-a94a-3f3e0892379b\") " pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:20.567924 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:20.567760 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:20.567924 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:20.567820 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:14:20.567924 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:20.567896 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:14:20.568059 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:20.567927 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs podName:f63deb5a-7f0d-4cca-a94a-3f3e0892379b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:36.56790462 +0000 UTC m=+34.246465933 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs") pod "network-metrics-daemon-6l86h" (UID: "f63deb5a-7f0d-4cca-a94a-3f3e0892379b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:14:20.568059 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:20.567955 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret podName:730e3c49-83b7-4a1d-9b7c-d5621955f6df nodeName:}" failed. No retries permitted until 2026-07-07 13:14:36.567937723 +0000 UTC m=+34.246499047 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret") pod "global-pull-secret-syncer-j2rqx" (UID: "730e3c49-83b7-4a1d-9b7c-d5621955f6df") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:14:20.668587 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:20.668532 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zxrv\" (UniqueName: \"kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv\") pod \"network-check-target-7cw9x\" (UID: \"6037cecb-d996-4bd8-a3d3-733a764f5163\") " pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:20.668745 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:20.668724 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:14:20.668803 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:20.668753 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:14:20.668803 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:20.668768 2568 projected.go:196] Error preparing data for projected volume kube-api-access-9zxrv for pod openshift-network-diagnostics/network-check-target-7cw9x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:20.668880 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:20.668828 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv podName:6037cecb-d996-4bd8-a3d3-733a764f5163 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:36.668813 +0000 UTC m=+34.347374310 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9zxrv" (UniqueName: "kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv") pod "network-check-target-7cw9x" (UID: "6037cecb-d996-4bd8-a3d3-733a764f5163") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:21.013071 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:21.013036 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:21.057372 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:21.057340 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" event={"ID":"a1ee8660983e09b5aeea7a0e782f81de","Type":"ContainerStarted","Data":"02297daccd272aab30ff4127265cfab2bcce6010ea82a3977e66260379da67b2"} Jul 07 13:14:21.059530 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:21.059513 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hm7dm" Jul 07 13:14:21.060487 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:21.060470 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hm7dm" Jul 07 13:14:21.068746 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:21.068724 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 13:14:21.076298 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:21.076254 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" podStartSLOduration=17.076241621 podStartE2EDuration="17.076241621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:14:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:21.076064148 +0000 UTC m=+18.754625481" watchObservedRunningTime="2026-07-07 13:14:21.076241621 +0000 UTC m=+18.754802952" Jul 07 13:14:21.087508 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:21.087486 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:21.189951 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:21.189867 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:21.806811 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:21.806706 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T13:14:21.068741207Z","UUID":"ac6f38bb-85f9-4e0e-814b-0ba8c94ee7bd","Handler":null,"Name":"","Endpoint":""} Jul 07 13:14:21.808294 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:21.808269 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 13:14:21.808427 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:21.808313 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 13:14:21.964737 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:21.964700 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:21.964737 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:21.964737 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:21.964958 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:21.964804 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:21.964958 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:21.964826 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:21.964958 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:21.964893 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:21.965094 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:21.965004 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:22.063191 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:22.063098 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" event={"ID":"54e31e35-d1da-4cf9-a0be-beda43aeb2bb","Type":"ContainerStarted","Data":"b52df7f486c20a21db4b76f7008e25824ebe2e9220a495308d2f9821e7c2a697"} Jul 07 13:14:22.573328 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:22.573290 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q9ck9_7416c196-b925-447d-8a35-a5c5421d15d1/dns-node-resolver/0.log" Jul 07 13:14:23.574130 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:23.574099 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z856v_0bbd3146-8a83-4cdd-9b41-410b53b540f4/node-ca/0.log" Jul 07 13:14:23.964991 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:23.964896 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:23.964991 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:23.964928 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:23.964991 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:23.964928 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:23.965263 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:23.965023 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:23.965263 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:23.965117 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:23.965263 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:23.965186 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:25.076667 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:25.076585 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:25.171500 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:25.171470 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:25.271851 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:25.271794 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:25.818446 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:25.818410 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:25.918493 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:25.918461 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:25.964713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:25.964678 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:25.964713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:25.964705 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:25.964713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:25.964719 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:25.964998 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:25.964813 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:25.964998 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:25.964936 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:25.965107 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:25.965039 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:27.075937 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:27.075610 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" event={"ID":"fbee5bee-337d-40af-baef-deee6657a83a","Type":"ContainerStarted","Data":"cf4962bc6b08134295bad2bd09af372a1932f6c7219f2ac66343e38145d6a3a9"} Jul 07 13:14:27.076617 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:27.075944 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" event={"ID":"fbee5bee-337d-40af-baef-deee6657a83a","Type":"ContainerStarted","Data":"f31a51515c39f866df62c8d6bc2be56a038f06f4f08574106769ad4280bba3f0"} Jul 07 13:14:27.077964 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:27.077930 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" event={"ID":"54e31e35-d1da-4cf9-a0be-beda43aeb2bb","Type":"ContainerStarted","Data":"d5ef5a57d472ea28d285bb0393d4e3b67c579a999108e369420138d20af6d09c"} Jul 07 13:14:27.108215 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:27.108158 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qd87" podStartSLOduration=3.117552618 podStartE2EDuration="24.108138657s" totalImagesPullingTime="20.990586039s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:14:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:27.107403845 +0000 UTC m=+24.785965175" watchObservedRunningTime="2026-07-07 13:14:27.108138657 +0000 UTC m=+24.786699989" Jul 07 13:14:27.766284 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:27.766249 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:27.828652 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:27.828616 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:27.965507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:27.965471 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:27.965692 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:27.965519 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:27.965692 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:27.965489 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:27.965692 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:27.965630 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:27.965845 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:27.965717 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:27.965845 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:27.965832 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:27.990047 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:27.990015 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:28.453390 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:28.453363 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:28.556483 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:28.556457 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:28.895324 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:28.895280 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:28.970136 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:28.970060 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:29.084767 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:29.084733 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" event={"ID":"fbee5bee-337d-40af-baef-deee6657a83a","Type":"ContainerStarted","Data":"48a958c3698afaa6d8bb4f3cdbff1ff6a8c9b48c7d219ae281bcef317d82f249"} Jul 07 13:14:29.084976 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:29.084802 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" event={"ID":"fbee5bee-337d-40af-baef-deee6657a83a","Type":"ContainerStarted","Data":"a0f30ea239f74e3f97c935ddb1c91c05c371e87b24c1ad6ff11c446f64273afa"} Jul 07 13:14:29.084976 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:29.084844 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" event={"ID":"fbee5bee-337d-40af-baef-deee6657a83a","Type":"ContainerStarted","Data":"130b5de73a6ccc6a39b7de8c4d305ada59d8fd0567baee3289d952dd7a4d48cd"} Jul 07 13:14:29.084976 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:29.084928 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" event={"ID":"fbee5bee-337d-40af-baef-deee6657a83a","Type":"ContainerStarted","Data":"677578c357a82e7e42d35892875037bb0d129305e80e279a6e06f862dcbe91f2"} Jul 07 13:14:29.086330 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:29.086308 2568 generic.go:356] "Generic (PLEG): container finished" podID="ab6cbacd-df97-40ea-9abd-e52a4f49af4d" containerID="1d19e6ca298dc65bcd90dd8a041ccbe2f7b7da405372f88f71be05ef47974bc6" exitCode=0 Jul 07 13:14:29.086436 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:29.086340 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h8rgd" event={"ID":"ab6cbacd-df97-40ea-9abd-e52a4f49af4d","Type":"ContainerDied","Data":"1d19e6ca298dc65bcd90dd8a041ccbe2f7b7da405372f88f71be05ef47974bc6"} Jul 07 13:14:29.965253 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:29.965220 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:29.965975 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:29.965223 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:29.965975 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:29.965351 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:29.965975 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:29.965220 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:29.965975 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:29.965420 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:29.965975 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:29.965517 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:30.163306 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:30.163279 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:30.264325 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:30.264292 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:31.093395 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:31.093358 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" event={"ID":"fbee5bee-337d-40af-baef-deee6657a83a","Type":"ContainerStarted","Data":"bdeee3db105eca8478e409b2ceee9be234fc121c634cdf46a0f0955a79a3b427"} Jul 07 13:14:31.095307 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:31.095286 2568 generic.go:356] "Generic (PLEG): container finished" podID="ab6cbacd-df97-40ea-9abd-e52a4f49af4d" containerID="a2f3647c0640dd09088440d696bc49fa0cdefe9eb5adb3b2b20932413071aa20" exitCode=0 Jul 07 13:14:31.095415 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:31.095314 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h8rgd" event={"ID":"ab6cbacd-df97-40ea-9abd-e52a4f49af4d","Type":"ContainerDied","Data":"a2f3647c0640dd09088440d696bc49fa0cdefe9eb5adb3b2b20932413071aa20"} Jul 07 13:14:31.304820 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:31.304791 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:31.701307 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:31.701270 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:31.965379 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:31.965308 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:31.965522 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:31.965308 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:31.965522 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:31.965419 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:31.965522 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:31.965485 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:31.965522 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:31.965308 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:31.965716 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:31.965614 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:31.999235 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:31.999208 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:32.107184 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:32.107157 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:32.206623 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:32.206590 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:32.363788 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:32.363754 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:33.102188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:33.102036 2568 generic.go:356] "Generic (PLEG): container finished" podID="ab6cbacd-df97-40ea-9abd-e52a4f49af4d" containerID="22a65b7cf38bba5bb9f96398921bd2186f22a58215fa6350b18999fcf3ec08a7" exitCode=0 Jul 07 13:14:33.102309 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:33.102099 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h8rgd" event={"ID":"ab6cbacd-df97-40ea-9abd-e52a4f49af4d","Type":"ContainerDied","Data":"22a65b7cf38bba5bb9f96398921bd2186f22a58215fa6350b18999fcf3ec08a7"} Jul 07 13:14:33.367812 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:33.367696 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:33.577929 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:33.577899 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:33.964862 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:33.964825 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:33.964862 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:33.964851 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:33.965119 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:33.964939 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:33.965119 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:33.964997 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:33.965224 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:33.965132 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:33.965268 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:33.965222 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:33.972594 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:33.972561 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:34.107538 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:34.107510 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" event={"ID":"fbee5bee-337d-40af-baef-deee6657a83a","Type":"ContainerStarted","Data":"799ea24662a2f8aa2029bee163382fc3bdf5edf7f9c076dfc98e394103051a85"} Jul 07 13:14:34.107783 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:34.107754 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:34.107930 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:34.107837 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:34.123338 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:34.123316 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:34.142618 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:34.142550 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" podStartSLOduration=9.55077412 podStartE2EDuration="31.142535817s" totalImagesPullingTime="21.591761697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:34.142361802 +0000 UTC m=+31.820923133" watchObservedRunningTime="2026-07-07 13:14:34.142535817 +0000 UTC m=+31.821097148" Jul 07 13:14:34.154977 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:34.154950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:34.231319 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:34.231234 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:34.493491 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:34.493412 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:34.597860 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:34.597775 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:34.767402 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:34.767306 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:35.110984 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:35.110898 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:35.127281 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:35.127257 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:14:35.161833 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:35.161802 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:35.366570 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:35.366484 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:35.465392 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:35.465359 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:35.673240 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:35.673211 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:35.776326 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:35.776294 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:35.965440 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:35.965358 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:35.965440 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:35.965385 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:35.965440 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:35.965385 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:35.965714 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:35.965446 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:35.965714 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:35.965539 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:35.965714 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:35.965649 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:36.154409 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:36.154377 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:36.255231 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:36.255151 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:36.593879 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:36.593796 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:36.593879 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:36.593837 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs\") pod \"network-metrics-daemon-6l86h\" (UID: \"f63deb5a-7f0d-4cca-a94a-3f3e0892379b\") " pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:36.594051 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:36.593939 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:14:36.594051 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:36.593950 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:14:36.594051 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:36.593997 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs podName:f63deb5a-7f0d-4cca-a94a-3f3e0892379b nodeName:}" failed. No retries permitted until 2026-07-07 13:15:08.593983339 +0000 UTC m=+66.272544649 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs") pod "network-metrics-daemon-6l86h" (UID: "f63deb5a-7f0d-4cca-a94a-3f3e0892379b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:14:36.594051 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:36.594010 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret podName:730e3c49-83b7-4a1d-9b7c-d5621955f6df nodeName:}" failed. No retries permitted until 2026-07-07 13:15:08.594004349 +0000 UTC m=+66.272565658 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret") pod "global-pull-secret-syncer-j2rqx" (UID: "730e3c49-83b7-4a1d-9b7c-d5621955f6df") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:14:36.694331 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:36.694295 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zxrv\" (UniqueName: \"kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv\") pod \"network-check-target-7cw9x\" (UID: \"6037cecb-d996-4bd8-a3d3-733a764f5163\") " pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:36.694766 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:36.694481 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:14:36.694766 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:36.694505 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:14:36.694766 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:36.694518 2568 projected.go:196] Error preparing data for projected volume kube-api-access-9zxrv for pod openshift-network-diagnostics/network-check-target-7cw9x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:36.694766 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:36.694598 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv podName:6037cecb-d996-4bd8-a3d3-733a764f5163 nodeName:}" failed. No retries permitted until 2026-07-07 13:15:08.694561455 +0000 UTC m=+66.373122778 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9zxrv" (UniqueName: "kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv") pod "network-check-target-7cw9x" (UID: "6037cecb-d996-4bd8-a3d3-733a764f5163") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:36.954608 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:36.954554 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:37.549726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:37.549692 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:37.792769 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:37.792731 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:37.871467 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:37.871371 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:37.954671 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:37.954641 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:37.964628 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:37.964609 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:37.964717 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:37.964609 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:37.964781 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:37.964708 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:37.964781 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:37.964612 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:37.964887 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:37.964772 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:37.964932 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:37.964881 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:38.054924 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:38.054890 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:38.760436 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:38.760405 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:39.354805 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:39.354766 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:39.565396 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:39.565360 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:39.780914 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:39.780872 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:39.959249 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:39.959221 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:39.965269 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:39.965253 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:39.965350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:39.965339 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:39.965400 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:39.965354 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:39.965456 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:39.965440 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:39.965486 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:39.965477 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:39.965544 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:39.965532 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:40.952799 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:40.952614 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:41.126294 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:41.126258 2568 generic.go:356] "Generic (PLEG): container finished" podID="ab6cbacd-df97-40ea-9abd-e52a4f49af4d" containerID="4e668be689d330a0f5f4d3191fcc99ea08fe44d3b0fa10fe651e4d910c461d5e" exitCode=0 Jul 07 13:14:41.126490 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:41.126293 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h8rgd" event={"ID":"ab6cbacd-df97-40ea-9abd-e52a4f49af4d","Type":"ContainerDied","Data":"4e668be689d330a0f5f4d3191fcc99ea08fe44d3b0fa10fe651e4d910c461d5e"} Jul 07 13:14:41.356956 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:41.356930 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:41.559700 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:41.559608 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:41.756228 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:41.756181 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:41.948955 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:41.948925 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:41.965121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:41.965088 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:41.965475 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:41.965092 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:41.965475 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:41.965192 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:41.965475 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:41.965234 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:41.965475 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:41.965334 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:41.965475 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:41.965426 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:42.132288 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:42.132255 2568 generic.go:356] "Generic (PLEG): container finished" podID="ab6cbacd-df97-40ea-9abd-e52a4f49af4d" containerID="ce13e88a5069f293c75e93b66cf63dcfbcc632fdb154390acb745eac64c6d0bc" exitCode=0 Jul 07 13:14:42.132451 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:42.132309 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h8rgd" event={"ID":"ab6cbacd-df97-40ea-9abd-e52a4f49af4d","Type":"ContainerDied","Data":"ce13e88a5069f293c75e93b66cf63dcfbcc632fdb154390acb745eac64c6d0bc"} Jul 07 13:14:42.350696 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:42.350670 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:43.126691 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:43.126655 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:43.137427 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:43.137399 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h8rgd" event={"ID":"ab6cbacd-df97-40ea-9abd-e52a4f49af4d","Type":"ContainerStarted","Data":"073ab57bdb8b697b8bf8f76100175dd43906309251562093a425f75b1a8639e2"} Jul 07 13:14:43.163225 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:43.163185 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-h8rgd" podStartSLOduration=11.754560188 podStartE2EDuration="40.163171331s" totalImagesPullingTime="28.408611143s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:14:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:43.162787797 +0000 UTC m=+40.841349127" watchObservedRunningTime="2026-07-07 13:14:43.163171331 +0000 UTC m=+40.841732661" Jul 07 13:14:43.227484 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:43.227448 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:43.328807 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:43.328771 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:43.428294 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:43.428268 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:43.615532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:43.615507 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:43.798362 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:43.798295 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:43.965007 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:43.964970 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:43.965007 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:43.964991 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:43.965007 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:43.965013 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:43.965325 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:43.965075 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:43.965325 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:43.965200 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:43.965325 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:43.965297 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:44.009302 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:44.009271 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:44.193957 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:44.193932 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:44.256944 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:44.256920 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:44.664614 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:44.664586 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:44.796985 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:44.796958 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:44.898610 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:44.898564 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:45.092544 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:45.092470 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:45.159221 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:45.159193 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:45.964940 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:45.964908 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:45.964940 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:45.964943 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:45.965374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:45.964947 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:45.965374 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:45.965034 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:45.965374 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:45.965112 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:45.965374 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:45.965184 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:47.965189 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:47.965142 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:47.965640 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:47.965142 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:47.965640 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:47.965248 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:47.965640 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:47.965142 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:47.965640 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:47.965290 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:47.965640 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:47.965389 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:49.964915 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:49.964885 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:49.965285 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:49.964885 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:49.965285 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:49.964983 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:49.965285 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:49.964885 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:49.965285 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:49.965045 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:49.965285 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:49.965124 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:51.185148 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:51.185096 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:51.288899 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:51.288868 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:51.387023 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:51.386994 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:51.485859 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:51.485779 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:51.965210 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:51.965177 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:51.965210 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:51.965215 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:51.965466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:51.965177 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:51.965466 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:51.965301 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:51.965466 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:51.965357 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:51.965466 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:51.965413 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:53.286235 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:53.286201 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:53.470097 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:53.470066 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:53.573377 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:53.573302 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:53.964832 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:53.964793 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:53.964832 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:53.964813 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:53.965067 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:53.964828 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:53.965067 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:53.964911 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:53.965067 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:53.964965 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:53.965197 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:53.965075 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:54.597314 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:54.597282 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7cw9x"] Jul 07 13:14:54.597970 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:54.597419 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:54.597970 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:54.597498 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:54.601245 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:54.601214 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j2rqx"] Jul 07 13:14:54.601384 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:54.601355 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:54.601521 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:54.601478 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:54.601851 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:54.601831 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6l86h"] Jul 07 13:14:54.601946 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:54.601930 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:54.602068 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:54.602028 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:55.965486 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:55.965302 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:55.966022 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:55.965302 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:55.966022 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:55.965601 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:55.966022 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:55.965303 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:55.966022 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:55.965639 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:55.966022 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:55.965683 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:57.632626 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:57.632592 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:57.708556 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:57.708525 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:57.965267 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:57.965231 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:57.965267 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:57.965252 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:57.965514 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:57.965338 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:57.965514 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:57.965341 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2rqx" podUID="730e3c49-83b7-4a1d-9b7c-d5621955f6df" Jul 07 13:14:57.965514 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:57.965433 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6l86h" podUID="f63deb5a-7f0d-4cca-a94a-3f3e0892379b" Jul 07 13:14:57.965645 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:57.965515 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cw9x" podUID="6037cecb-d996-4bd8-a3d3-733a764f5163" Jul 07 13:14:58.566783 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.566760 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeReady" Jul 07 13:14:58.566946 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.566883 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 13:14:58.619181 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.619100 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qzrm2"] Jul 07 13:14:58.621361 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.621337 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qzrm2" Jul 07 13:14:58.624276 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.624257 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 13:14:58.624590 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.624557 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 13:14:58.624693 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.624667 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qwjzx\"" Jul 07 13:14:58.631280 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.631263 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-25qdm"] Jul 07 13:14:58.633164 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.633152 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:14:58.647284 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.647258 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qzrm2"] Jul 07 13:14:58.647429 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:58.647403 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"insights-runtime-extractor-tls\" is forbidden: User \"system:node:ip-10-0-136-215.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-136-215.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" type="*v1.Secret" Jul 07 13:14:58.647628 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:58.647609 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"insights-runtime-extractor-sa-dockercfg-wkhkp\" is forbidden: User \"system:node:ip-10-0-136-215.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-136-215.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wkhkp\"" type="*v1.Secret" Jul 07 13:14:58.647693 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:58.647637 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-rbac-proxy\" is forbidden: User \"system:node:ip-10-0-136-215.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-136-215.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" type="*v1.ConfigMap" Jul 07 13:14:58.647916 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:58.647897 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-136-215.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-136-215.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 07 13:14:58.647916 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:58.647904 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-136-215.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-136-215.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 07 13:14:58.655225 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.655205 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/faf07002-635a-4f2b-a452-be3e757a9d2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:14:58.655312 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.655257 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e1f2bac-745c-46b5-8a4b-b745449eb8e2-metrics-tls\") pod \"dns-default-qzrm2\" (UID: \"7e1f2bac-745c-46b5-8a4b-b745449eb8e2\") " pod="openshift-dns/dns-default-qzrm2" Jul 07 13:14:58.655312 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.655281 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e1f2bac-745c-46b5-8a4b-b745449eb8e2-tmp-dir\") pod \"dns-default-qzrm2\" (UID: \"7e1f2bac-745c-46b5-8a4b-b745449eb8e2\") " pod="openshift-dns/dns-default-qzrm2" Jul 07 13:14:58.655312 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.655301 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7e1f2bac-745c-46b5-8a4b-b745449eb8e2-config-volume\") pod \"dns-default-qzrm2\" (UID: \"7e1f2bac-745c-46b5-8a4b-b745449eb8e2\") " pod="openshift-dns/dns-default-qzrm2" Jul 07 13:14:58.655420 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.655319 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/faf07002-635a-4f2b-a452-be3e757a9d2f-data-volume\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:14:58.655420 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.655335 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/faf07002-635a-4f2b-a452-be3e757a9d2f-crio-socket\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:14:58.655420 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.655351 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-272cb\" (UniqueName: \"kubernetes.io/projected/faf07002-635a-4f2b-a452-be3e757a9d2f-kube-api-access-272cb\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:14:58.655420 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.655367 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqxgq\" (UniqueName: \"kubernetes.io/projected/7e1f2bac-745c-46b5-8a4b-b745449eb8e2-kube-api-access-pqxgq\") pod \"dns-default-qzrm2\" (UID: \"7e1f2bac-745c-46b5-8a4b-b745449eb8e2\") " pod="openshift-dns/dns-default-qzrm2" Jul 07 13:14:58.655420 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.655385 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/faf07002-635a-4f2b-a452-be3e757a9d2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:14:58.675418 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.675393 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-25qdm"] Jul 07 13:14:58.741623 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.741598 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9zqg5"] Jul 07 13:14:58.743780 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.743766 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9zqg5" Jul 07 13:14:58.747436 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.747418 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mbgxn\"" Jul 07 13:14:58.748102 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.748080 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 13:14:58.748177 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.748106 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 13:14:58.748315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.748304 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 13:14:58.755764 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.755746 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e1f2bac-745c-46b5-8a4b-b745449eb8e2-metrics-tls\") pod \"dns-default-qzrm2\" (UID: \"7e1f2bac-745c-46b5-8a4b-b745449eb8e2\") " pod="openshift-dns/dns-default-qzrm2" Jul 07 13:14:58.755848 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.755769 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e1f2bac-745c-46b5-8a4b-b745449eb8e2-tmp-dir\") pod \"dns-default-qzrm2\" (UID: \"7e1f2bac-745c-46b5-8a4b-b745449eb8e2\") " pod="openshift-dns/dns-default-qzrm2" Jul 07 13:14:58.755848 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.755794 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7e1f2bac-745c-46b5-8a4b-b745449eb8e2-config-volume\") pod \"dns-default-qzrm2\" (UID: \"7e1f2bac-745c-46b5-8a4b-b745449eb8e2\") " pod="openshift-dns/dns-default-qzrm2" Jul 07 13:14:58.755848 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.755810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/faf07002-635a-4f2b-a452-be3e757a9d2f-data-volume\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:14:58.755848 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.755841 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/faf07002-635a-4f2b-a452-be3e757a9d2f-crio-socket\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:14:58.756061 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.755865 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-272cb\" (UniqueName: \"kubernetes.io/projected/faf07002-635a-4f2b-a452-be3e757a9d2f-kube-api-access-272cb\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:14:58.756061 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.755981 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqxgq\" (UniqueName: \"kubernetes.io/projected/7e1f2bac-745c-46b5-8a4b-b745449eb8e2-kube-api-access-pqxgq\") pod \"dns-default-qzrm2\" (UID: \"7e1f2bac-745c-46b5-8a4b-b745449eb8e2\") " pod="openshift-dns/dns-default-qzrm2" Jul 07 13:14:58.756061 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.756038 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/faf07002-635a-4f2b-a452-be3e757a9d2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:14:58.756194 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.756059 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/faf07002-635a-4f2b-a452-be3e757a9d2f-crio-socket\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:14:58.756194 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.756092 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/faf07002-635a-4f2b-a452-be3e757a9d2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:14:58.756194 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.756153 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e1f2bac-745c-46b5-8a4b-b745449eb8e2-tmp-dir\") pod \"dns-default-qzrm2\" (UID: \"7e1f2bac-745c-46b5-8a4b-b745449eb8e2\") " pod="openshift-dns/dns-default-qzrm2" Jul 07 13:14:58.756313 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.756200 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/faf07002-635a-4f2b-a452-be3e757a9d2f-data-volume\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:14:58.756437 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.756419 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7e1f2bac-745c-46b5-8a4b-b745449eb8e2-config-volume\") pod \"dns-default-qzrm2\" (UID: \"7e1f2bac-745c-46b5-8a4b-b745449eb8e2\") " pod="openshift-dns/dns-default-qzrm2" Jul 07 13:14:58.760100 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.760076 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e1f2bac-745c-46b5-8a4b-b745449eb8e2-metrics-tls\") pod \"dns-default-qzrm2\" (UID: \"7e1f2bac-745c-46b5-8a4b-b745449eb8e2\") " pod="openshift-dns/dns-default-qzrm2" Jul 07 13:14:58.765693 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.765672 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9zqg5"] Jul 07 13:14:58.777887 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.777867 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqxgq\" (UniqueName: \"kubernetes.io/projected/7e1f2bac-745c-46b5-8a4b-b745449eb8e2-kube-api-access-pqxgq\") pod \"dns-default-qzrm2\" (UID: \"7e1f2bac-745c-46b5-8a4b-b745449eb8e2\") " pod="openshift-dns/dns-default-qzrm2" Jul 07 13:14:58.856450 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.856420 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/627d53fa-7581-4f07-94ee-78932d124b75-cert\") pod \"ingress-canary-9zqg5\" (UID: \"627d53fa-7581-4f07-94ee-78932d124b75\") " pod="openshift-ingress-canary/ingress-canary-9zqg5" Jul 07 13:14:58.856655 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.856492 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t86kp\" (UniqueName: \"kubernetes.io/projected/627d53fa-7581-4f07-94ee-78932d124b75-kube-api-access-t86kp\") pod \"ingress-canary-9zqg5\" (UID: \"627d53fa-7581-4f07-94ee-78932d124b75\") " pod="openshift-ingress-canary/ingress-canary-9zqg5" Jul 07 13:14:58.857159 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.857136 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:58.929871 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.929846 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:58.930780 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.930765 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qzrm2" Jul 07 13:14:58.957772 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.957748 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t86kp\" (UniqueName: \"kubernetes.io/projected/627d53fa-7581-4f07-94ee-78932d124b75-kube-api-access-t86kp\") pod \"ingress-canary-9zqg5\" (UID: \"627d53fa-7581-4f07-94ee-78932d124b75\") " pod="openshift-ingress-canary/ingress-canary-9zqg5" Jul 07 13:14:58.957875 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.957801 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/627d53fa-7581-4f07-94ee-78932d124b75-cert\") pod \"ingress-canary-9zqg5\" (UID: \"627d53fa-7581-4f07-94ee-78932d124b75\") " pod="openshift-ingress-canary/ingress-canary-9zqg5" Jul 07 13:14:58.960221 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.960199 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/627d53fa-7581-4f07-94ee-78932d124b75-cert\") pod \"ingress-canary-9zqg5\" (UID: \"627d53fa-7581-4f07-94ee-78932d124b75\") " pod="openshift-ingress-canary/ingress-canary-9zqg5" Jul 07 13:14:58.966911 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:58.966890 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t86kp\" (UniqueName: \"kubernetes.io/projected/627d53fa-7581-4f07-94ee-78932d124b75-kube-api-access-t86kp\") pod \"ingress-canary-9zqg5\" (UID: \"627d53fa-7581-4f07-94ee-78932d124b75\") " pod="openshift-ingress-canary/ingress-canary-9zqg5" Jul 07 13:14:59.022128 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.022106 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-866f955bdc-g8wjb"] Jul 07 13:14:59.028842 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.028821 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.031803 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.031735 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dkwl9\"" Jul 07 13:14:59.031803 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.031770 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 13:14:59.031803 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.031771 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 13:14:59.032059 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.031844 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 13:14:59.032059 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.031737 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 13:14:59.032156 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.032096 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 13:14:59.032156 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.032110 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 13:14:59.032428 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.032413 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 13:14:59.036402 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.036378 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-866f955bdc-g8wjb"] Jul 07 13:14:59.037751 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.037733 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 13:14:59.052242 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.052222 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9zqg5" Jul 07 13:14:59.058632 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.058613 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-trusted-ca-bundle\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.058757 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.058653 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b69addc-9aea-42aa-a333-db21b736ba26-console-serving-cert\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.058757 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.058679 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-console-config\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.058757 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.058723 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fptbd\" (UniqueName: \"kubernetes.io/projected/3b69addc-9aea-42aa-a333-db21b736ba26-kube-api-access-fptbd\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.058849 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.058820 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-service-ca\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.058849 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.058840 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-oauth-serving-cert\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.058910 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.058864 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b69addc-9aea-42aa-a333-db21b736ba26-console-oauth-config\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.066354 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.066333 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qzrm2"] Jul 07 13:14:59.069213 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:59.069193 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e1f2bac_745c_46b5_8a4b_b745449eb8e2.slice/crio-49360c63915b1b7af4fe0ddbee49bfde05b210ea1272a81df5f2a748e57f358e WatchSource:0}: Error finding container 49360c63915b1b7af4fe0ddbee49bfde05b210ea1272a81df5f2a748e57f358e: Status 404 returned error can't find the container with id 49360c63915b1b7af4fe0ddbee49bfde05b210ea1272a81df5f2a748e57f358e Jul 07 13:14:59.159850 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.159817 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b69addc-9aea-42aa-a333-db21b736ba26-console-serving-cert\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.160004 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.159854 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-console-config\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.160004 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.159897 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptbd\" (UniqueName: \"kubernetes.io/projected/3b69addc-9aea-42aa-a333-db21b736ba26-kube-api-access-fptbd\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.160004 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.159945 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-service-ca\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.160004 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.159960 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-oauth-serving-cert\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.160004 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.159983 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b69addc-9aea-42aa-a333-db21b736ba26-console-oauth-config\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.160235 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.160014 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-trusted-ca-bundle\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.160787 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.160757 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-service-ca\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.161004 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.160980 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-oauth-serving-cert\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.161140 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.161076 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-console-config\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.161314 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.161296 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-trusted-ca-bundle\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.162982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.162960 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b69addc-9aea-42aa-a333-db21b736ba26-console-serving-cert\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.163044 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.162986 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b69addc-9aea-42aa-a333-db21b736ba26-console-oauth-config\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.170743 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.170723 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qzrm2" event={"ID":"7e1f2bac-745c-46b5-8a4b-b745449eb8e2","Type":"ContainerStarted","Data":"49360c63915b1b7af4fe0ddbee49bfde05b210ea1272a81df5f2a748e57f358e"} Jul 07 13:14:59.173027 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.173005 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fptbd\" (UniqueName: \"kubernetes.io/projected/3b69addc-9aea-42aa-a333-db21b736ba26-kube-api-access-fptbd\") pod \"console-866f955bdc-g8wjb\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.175185 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.175164 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9zqg5"] Jul 07 13:14:59.178719 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:59.178700 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod627d53fa_7581_4f07_94ee_78932d124b75.slice/crio-465f3100b1f693438678a8623c94c442bc45175109f17f490319de08c10fb056 WatchSource:0}: Error finding container 465f3100b1f693438678a8623c94c442bc45175109f17f490319de08c10fb056: Status 404 returned error can't find the container with id 465f3100b1f693438678a8623c94c442bc45175109f17f490319de08c10fb056 Jul 07 13:14:59.338353 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.338314 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:14:59.481462 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.481387 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-866f955bdc-g8wjb"] Jul 07 13:14:59.484665 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:14:59.484638 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b69addc_9aea_42aa_a333_db21b736ba26.slice/crio-3f7c0cfef5d34c79fcaf94506f0383909a8e58dacec7c32b51f04051e1daa75b WatchSource:0}: Error finding container 3f7c0cfef5d34c79fcaf94506f0383909a8e58dacec7c32b51f04051e1daa75b: Status 404 returned error can't find the container with id 3f7c0cfef5d34c79fcaf94506f0383909a8e58dacec7c32b51f04051e1daa75b Jul 07 13:14:59.525175 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.525149 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 13:14:59.678463 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.678217 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 13:14:59.688909 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.688840 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-272cb\" (UniqueName: \"kubernetes.io/projected/faf07002-635a-4f2b-a452-be3e757a9d2f-kube-api-access-272cb\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:14:59.756271 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:59.756238 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: failed to sync configmap cache: timed out waiting for the condition Jul 07 13:14:59.756434 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:59.756418 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/faf07002-635a-4f2b-a452-be3e757a9d2f-kube-rbac-proxy-cm podName:faf07002-635a-4f2b-a452-be3e757a9d2f nodeName:}" failed. No retries permitted until 2026-07-07 13:15:00.256327522 +0000 UTC m=+57.934888838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/faf07002-635a-4f2b-a452-be3e757a9d2f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-25qdm" (UID: "faf07002-635a-4f2b-a452-be3e757a9d2f") : failed to sync configmap cache: timed out waiting for the condition Jul 07 13:14:59.756519 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:59.756438 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: failed to sync secret cache: timed out waiting for the condition Jul 07 13:14:59.756519 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:14:59.756499 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf07002-635a-4f2b-a452-be3e757a9d2f-insights-runtime-extractor-tls podName:faf07002-635a-4f2b-a452-be3e757a9d2f nodeName:}" failed. No retries permitted until 2026-07-07 13:15:00.256483288 +0000 UTC m=+57.935044598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/faf07002-635a-4f2b-a452-be3e757a9d2f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-25qdm" (UID: "faf07002-635a-4f2b-a452-be3e757a9d2f") : failed to sync secret cache: timed out waiting for the condition Jul 07 13:14:59.811161 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.811136 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wkhkp\"" Jul 07 13:14:59.950318 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.950216 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 13:14:59.965169 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.965140 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:14:59.965169 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.965170 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:14:59.965453 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.965436 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:14:59.968351 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.968324 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 13:14:59.968699 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.968683 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 13:14:59.968802 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.968711 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 13:14:59.968802 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.968685 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 13:14:59.969312 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.969293 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fjddr\"" Jul 07 13:14:59.969477 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:14:59.969446 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5ztml\"" Jul 07 13:15:00.029522 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:00.029498 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 13:15:00.175637 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:00.175598 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9zqg5" event={"ID":"627d53fa-7581-4f07-94ee-78932d124b75","Type":"ContainerStarted","Data":"465f3100b1f693438678a8623c94c442bc45175109f17f490319de08c10fb056"} Jul 07 13:15:00.177161 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:00.177121 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-866f955bdc-g8wjb" event={"ID":"3b69addc-9aea-42aa-a333-db21b736ba26","Type":"ContainerStarted","Data":"3f7c0cfef5d34c79fcaf94506f0383909a8e58dacec7c32b51f04051e1daa75b"} Jul 07 13:15:00.268384 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:00.268303 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/faf07002-635a-4f2b-a452-be3e757a9d2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:15:00.268539 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:00.268372 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/faf07002-635a-4f2b-a452-be3e757a9d2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:15:00.269058 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:00.269019 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/faf07002-635a-4f2b-a452-be3e757a9d2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:15:00.271474 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:00.271449 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/faf07002-635a-4f2b-a452-be3e757a9d2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-25qdm\" (UID: \"faf07002-635a-4f2b-a452-be3e757a9d2f\") " pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:15:00.440425 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:00.440386 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-25qdm" Jul 07 13:15:00.638629 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:00.638598 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-25qdm"] Jul 07 13:15:00.642772 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:15:00.642741 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf07002_635a_4f2b_a452_be3e757a9d2f.slice/crio-4c6c3c52201e9429e20160508175d47d10c46b9c1902a9a480e9a8f4293152c1 WatchSource:0}: Error finding container 4c6c3c52201e9429e20160508175d47d10c46b9c1902a9a480e9a8f4293152c1: Status 404 returned error can't find the container with id 4c6c3c52201e9429e20160508175d47d10c46b9c1902a9a480e9a8f4293152c1 Jul 07 13:15:00.867081 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:00.866982 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:01.005679 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:01.005642 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:01.182307 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:01.182260 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-25qdm" event={"ID":"faf07002-635a-4f2b-a452-be3e757a9d2f","Type":"ContainerStarted","Data":"4c6c3c52201e9429e20160508175d47d10c46b9c1902a9a480e9a8f4293152c1"} Jul 07 13:15:01.216513 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:01.216469 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:01.316834 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:01.316798 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:02.412465 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:02.412425 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:02.496755 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:02.496708 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:04.179164 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.178942 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:04.192140 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.192102 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-866f955bdc-g8wjb" event={"ID":"3b69addc-9aea-42aa-a333-db21b736ba26","Type":"ContainerStarted","Data":"04725e0c3c4c582f2faeee8fe52c9ad2ff241954966baacbcb9d2d5a0acd9f83"} Jul 07 13:15:04.194213 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.194146 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9zqg5" event={"ID":"627d53fa-7581-4f07-94ee-78932d124b75","Type":"ContainerStarted","Data":"dd851abaf4bfb1744f1fdab1e4bc4a26e1dcf68042f718c2007d784b62febb41"} Jul 07 13:15:04.196188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.196166 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qzrm2" event={"ID":"7e1f2bac-745c-46b5-8a4b-b745449eb8e2","Type":"ContainerStarted","Data":"1d3837577e01061a45c7cb10dbeede38ce50d1b9b78f8ca18fedd29e526ba8be"} Jul 07 13:15:04.196328 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.196209 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qzrm2" event={"ID":"7e1f2bac-745c-46b5-8a4b-b745449eb8e2","Type":"ContainerStarted","Data":"660689f2756a5ee60b610dc812e9f36b1ded1c4b53ccd4d3fb4ebd4851e5e52f"} Jul 07 13:15:04.200070 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.197241 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qzrm2" Jul 07 13:15:04.212695 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.212632 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-866f955bdc-g8wjb" podStartSLOduration=1.245710846 podStartE2EDuration="5.212610983s" totalImagesPullingTime="3.966900137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:04.211838433 +0000 UTC m=+61.890399765" watchObservedRunningTime="2026-07-07 13:15:04.212610983 +0000 UTC m=+61.891172316" Jul 07 13:15:04.238699 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.238634 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qzrm2" podStartSLOduration=1.865396498 podStartE2EDuration="6.238612582s" totalImagesPullingTime="4.373216084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:04.237799471 +0000 UTC m=+61.916360803" watchObservedRunningTime="2026-07-07 13:15:04.238612582 +0000 UTC m=+61.917173915" Jul 07 13:15:04.259286 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.259227 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9zqg5" podStartSLOduration=1.9884340790000001 podStartE2EDuration="6.259207258s" totalImagesPullingTime="4.270773179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:04.258991394 +0000 UTC m=+61.937552726" watchObservedRunningTime="2026-07-07 13:15:04.259207258 +0000 UTC m=+61.937768589" Jul 07 13:15:04.272990 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.272953 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:04.420780 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.420746 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:04.453063 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.452991 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp"] Jul 07 13:15:04.456250 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.456227 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:04.460408 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.460385 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 13:15:04.460548 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.460516 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-zpcdd\"" Jul 07 13:15:04.461017 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.461000 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 13:15:04.461284 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.461247 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 13:15:04.461379 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.461326 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 13:15:04.461446 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.461248 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 13:15:04.478253 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.478228 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp"] Jul 07 13:15:04.506590 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.506547 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:04.506767 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.506749 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc0d4d82-c529-48b8-b7ef-156bec15b216-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-vqwpp\" (UID: \"fc0d4d82-c529-48b8-b7ef-156bec15b216\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:04.506830 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.506785 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fc0d4d82-c529-48b8-b7ef-156bec15b216-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-vqwpp\" (UID: \"fc0d4d82-c529-48b8-b7ef-156bec15b216\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:04.506830 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.506803 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dks59\" (UniqueName: \"kubernetes.io/projected/fc0d4d82-c529-48b8-b7ef-156bec15b216-kube-api-access-dks59\") pod \"prometheus-operator-567f7b8b7c-vqwpp\" (UID: \"fc0d4d82-c529-48b8-b7ef-156bec15b216\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:04.506937 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.506901 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc0d4d82-c529-48b8-b7ef-156bec15b216-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-vqwpp\" (UID: \"fc0d4d82-c529-48b8-b7ef-156bec15b216\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:04.607595 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.607545 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc0d4d82-c529-48b8-b7ef-156bec15b216-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-vqwpp\" (UID: \"fc0d4d82-c529-48b8-b7ef-156bec15b216\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:04.607791 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.607610 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fc0d4d82-c529-48b8-b7ef-156bec15b216-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-vqwpp\" (UID: \"fc0d4d82-c529-48b8-b7ef-156bec15b216\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:04.607791 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.607631 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dks59\" (UniqueName: \"kubernetes.io/projected/fc0d4d82-c529-48b8-b7ef-156bec15b216-kube-api-access-dks59\") pod \"prometheus-operator-567f7b8b7c-vqwpp\" (UID: \"fc0d4d82-c529-48b8-b7ef-156bec15b216\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:04.607791 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.607676 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc0d4d82-c529-48b8-b7ef-156bec15b216-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-vqwpp\" (UID: \"fc0d4d82-c529-48b8-b7ef-156bec15b216\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:04.607791 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:15:04.607775 2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 07 13:15:04.608016 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:15:04.607871 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc0d4d82-c529-48b8-b7ef-156bec15b216-prometheus-operator-tls podName:fc0d4d82-c529-48b8-b7ef-156bec15b216 nodeName:}" failed. No retries permitted until 2026-07-07 13:15:05.107850709 +0000 UTC m=+62.786412028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/fc0d4d82-c529-48b8-b7ef-156bec15b216-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-vqwpp" (UID: "fc0d4d82-c529-48b8-b7ef-156bec15b216") : secret "prometheus-operator-tls" not found Jul 07 13:15:04.608469 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.608436 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc0d4d82-c529-48b8-b7ef-156bec15b216-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-vqwpp\" (UID: \"fc0d4d82-c529-48b8-b7ef-156bec15b216\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:04.611479 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.611455 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc0d4d82-c529-48b8-b7ef-156bec15b216-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-vqwpp\" (UID: \"fc0d4d82-c529-48b8-b7ef-156bec15b216\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:04.620183 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.620156 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dks59\" (UniqueName: \"kubernetes.io/projected/fc0d4d82-c529-48b8-b7ef-156bec15b216-kube-api-access-dks59\") pod \"prometheus-operator-567f7b8b7c-vqwpp\" (UID: \"fc0d4d82-c529-48b8-b7ef-156bec15b216\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:04.797233 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:04.797116 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:05.112736 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:05.112641 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fc0d4d82-c529-48b8-b7ef-156bec15b216-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-vqwpp\" (UID: \"fc0d4d82-c529-48b8-b7ef-156bec15b216\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:05.115198 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:05.115168 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fc0d4d82-c529-48b8-b7ef-156bec15b216-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-vqwpp\" (UID: \"fc0d4d82-c529-48b8-b7ef-156bec15b216\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:05.367095 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:05.367021 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" Jul 07 13:15:05.862362 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:05.862322 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:05.944174 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:05.944108 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:06.793893 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:06.793870 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp"] Jul 07 13:15:06.796159 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:15:06.796136 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc0d4d82_c529_48b8_b7ef_156bec15b216.slice/crio-5db7b647d7dc329dbddfb8c21cd00784196798e686410f5266f95cff4a3f465c WatchSource:0}: Error finding container 5db7b647d7dc329dbddfb8c21cd00784196798e686410f5266f95cff4a3f465c: Status 404 returned error can't find the container with id 5db7b647d7dc329dbddfb8c21cd00784196798e686410f5266f95cff4a3f465c Jul 07 13:15:07.024741 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:07.024706 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:07.097216 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:07.097190 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:07.125716 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:07.125693 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8wjmr" Jul 07 13:15:07.208511 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:07.208475 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-25qdm" event={"ID":"faf07002-635a-4f2b-a452-be3e757a9d2f","Type":"ContainerStarted","Data":"4375592d8e0ad7f4d31b22755fe6462076393f0802e5f5fdff91d1fba8a29823"} Jul 07 13:15:07.209406 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:07.209383 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" event={"ID":"fc0d4d82-c529-48b8-b7ef-156bec15b216","Type":"ContainerStarted","Data":"5db7b647d7dc329dbddfb8c21cd00784196798e686410f5266f95cff4a3f465c"} Jul 07 13:15:07.259162 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:07.259130 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:07.350416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:07.350322 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:08.215981 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.215947 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-25qdm" event={"ID":"faf07002-635a-4f2b-a452-be3e757a9d2f","Type":"ContainerStarted","Data":"05ab0aa3df4279e638f94a5b51ab82fc2914fa0503b1812140722bcaa288be62"} Jul 07 13:15:08.643655 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.643570 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:15:08.643853 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.643667 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs\") pod \"network-metrics-daemon-6l86h\" (UID: \"f63deb5a-7f0d-4cca-a94a-3f3e0892379b\") " pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:15:08.646851 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.646666 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 13:15:08.647763 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.647728 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 13:15:08.656527 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.656455 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f63deb5a-7f0d-4cca-a94a-3f3e0892379b-metrics-certs\") pod \"network-metrics-daemon-6l86h\" (UID: \"f63deb5a-7f0d-4cca-a94a-3f3e0892379b\") " pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:15:08.656527 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.656455 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/730e3c49-83b7-4a1d-9b7c-d5621955f6df-original-pull-secret\") pod \"global-pull-secret-syncer-j2rqx\" (UID: \"730e3c49-83b7-4a1d-9b7c-d5621955f6df\") " pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:15:08.687250 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.687215 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2rqx" Jul 07 13:15:08.696444 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.696235 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fjddr\"" Jul 07 13:15:08.706073 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.704093 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6l86h" Jul 07 13:15:08.744335 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.744248 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zxrv\" (UniqueName: \"kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv\") pod \"network-check-target-7cw9x\" (UID: \"6037cecb-d996-4bd8-a3d3-733a764f5163\") " pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:15:08.747732 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.747478 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 13:15:08.755507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.755479 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:08.757485 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.757416 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 13:15:08.768819 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.768762 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zxrv\" (UniqueName: \"kubernetes.io/projected/6037cecb-d996-4bd8-a3d3-733a764f5163-kube-api-access-9zxrv\") pod \"network-check-target-7cw9x\" (UID: \"6037cecb-d996-4bd8-a3d3-733a764f5163\") " pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:15:08.846276 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.846242 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j2rqx"] Jul 07 13:15:08.849340 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:15:08.849311 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod730e3c49_83b7_4a1d_9b7c_d5621955f6df.slice/crio-a1f2fc6dcb2a00657c3b2961d35ef16c418c431dee67be97cca334a88a1f1055 WatchSource:0}: Error finding container a1f2fc6dcb2a00657c3b2961d35ef16c418c431dee67be97cca334a88a1f1055: Status 404 returned error can't find the container with id a1f2fc6dcb2a00657c3b2961d35ef16c418c431dee67be97cca334a88a1f1055 Jul 07 13:15:08.867668 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.867487 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6l86h"] Jul 07 13:15:08.870237 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:15:08.870201 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf63deb5a_7f0d_4cca_a94a_3f3e0892379b.slice/crio-30b0aff0411ef281e0d6e7ef3fe94d8c9095b7ef8d625afb9af0818f32092cde WatchSource:0}: Error finding container 30b0aff0411ef281e0d6e7ef3fe94d8c9095b7ef8d625afb9af0818f32092cde: Status 404 returned error can't find the container with id 30b0aff0411ef281e0d6e7ef3fe94d8c9095b7ef8d625afb9af0818f32092cde Jul 07 13:15:08.982091 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.982064 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5ztml\"" Jul 07 13:15:08.989651 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:08.989626 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:15:09.133740 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:09.133708 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7cw9x"] Jul 07 13:15:09.221423 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:09.221390 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" event={"ID":"fc0d4d82-c529-48b8-b7ef-156bec15b216","Type":"ContainerStarted","Data":"879db0b66bb6bffc1d7bd41f51123b1f30b91c221a7f97cea4b8a5db79e5c554"} Jul 07 13:15:09.221878 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:09.221453 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" event={"ID":"fc0d4d82-c529-48b8-b7ef-156bec15b216","Type":"ContainerStarted","Data":"e63eb7df4032d6d8c0d8c1c0912d00c87e2c49ed82ff0f344d6325a0b5467f13"} Jul 07 13:15:09.222631 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:09.222597 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6l86h" event={"ID":"f63deb5a-7f0d-4cca-a94a-3f3e0892379b","Type":"ContainerStarted","Data":"30b0aff0411ef281e0d6e7ef3fe94d8c9095b7ef8d625afb9af0818f32092cde"} Jul 07 13:15:09.223892 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:09.223871 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j2rqx" event={"ID":"730e3c49-83b7-4a1d-9b7c-d5621955f6df","Type":"ContainerStarted","Data":"a1f2fc6dcb2a00657c3b2961d35ef16c418c431dee67be97cca334a88a1f1055"} Jul 07 13:15:09.242063 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:09.242014 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vqwpp" podStartSLOduration=3.601410746 podStartE2EDuration="5.241997197s" totalImagesPullingTime="1.640586451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:09.241861092 +0000 UTC m=+66.920422423" watchObservedRunningTime="2026-07-07 13:15:09.241997197 +0000 UTC m=+66.920558527" Jul 07 13:15:09.339364 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:09.339265 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:15:09.339364 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:09.339339 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:15:09.344745 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:09.344720 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:15:10.032412 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.032367 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:10.104422 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.104381 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:10.233850 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.233825 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:15:10.320679 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:15:10.320593 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6037cecb_d996_4bd8_a3d3_733a764f5163.slice/crio-9eef3f3c6400d66b797908d7c6aed577a431632d5b053d5fc3fd23290ffe91fb WatchSource:0}: Error finding container 9eef3f3c6400d66b797908d7c6aed577a431632d5b053d5fc3fd23290ffe91fb: Status 404 returned error can't find the container with id 9eef3f3c6400d66b797908d7c6aed577a431632d5b053d5fc3fd23290ffe91fb Jul 07 13:15:10.405082 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.405055 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:10.506223 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.506188 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:10.643335 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.643262 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:10.741473 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.741446 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:10.843397 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.843328 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:10.926400 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.926366 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz"] Jul 07 13:15:10.929019 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.928983 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:10.930651 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.930456 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp"] Jul 07 13:15:10.932537 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.932515 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:10.936757 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.936240 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 13:15:10.936757 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.936490 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 13:15:10.936930 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.936808 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-kvvk5\"" Jul 07 13:15:10.937057 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.937034 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 13:15:10.937235 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.937211 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 13:15:10.937395 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.937236 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-65pgv\"" Jul 07 13:15:10.937395 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.937340 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 13:15:10.950766 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.950702 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz"] Jul 07 13:15:10.960054 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.958846 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7vrl2"] Jul 07 13:15:10.962967 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.961216 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:10.964692 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.964671 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-h2gvw\"" Jul 07 13:15:10.968896 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.968854 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 13:15:10.969280 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.969262 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 13:15:10.969364 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.969310 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:10.969433 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.969396 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 13:15:10.974292 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:10.972968 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp"] Jul 07 13:15:11.064820 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.064774 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffac38b9-3729-4e85-8128-ebf688f659fd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-xb8hz\" (UID: \"ffac38b9-3729-4e85-8128-ebf688f659fd\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:11.064985 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.064854 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.064985 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.064903 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-accelerators-collector-config\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.064985 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.064938 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffac38b9-3729-4e85-8128-ebf688f659fd-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-xb8hz\" (UID: \"ffac38b9-3729-4e85-8128-ebf688f659fd\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:11.064985 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.064963 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-tls\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.065168 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.064989 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.065168 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.065079 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffac38b9-3729-4e85-8128-ebf688f659fd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-xb8hz\" (UID: \"ffac38b9-3729-4e85-8128-ebf688f659fd\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:11.065168 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.065116 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d16107fb-6648-4083-ab38-dc0f21497f5a-metrics-client-ca\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.065168 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.065139 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bxh6\" (UniqueName: \"kubernetes.io/projected/d16107fb-6648-4083-ab38-dc0f21497f5a-kube-api-access-6bxh6\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.065324 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.065184 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-wtmp\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.065324 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.065214 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l55pc\" (UniqueName: \"kubernetes.io/projected/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-kube-api-access-l55pc\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.065324 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.065267 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.065324 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.065296 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-textfile\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.065497 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.065325 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.065497 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.065353 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.065497 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.065400 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n594p\" (UniqueName: \"kubernetes.io/projected/ffac38b9-3729-4e85-8128-ebf688f659fd-kube-api-access-n594p\") pod \"openshift-state-metrics-54cf96dfcf-xb8hz\" (UID: \"ffac38b9-3729-4e85-8128-ebf688f659fd\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:11.065497 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.065431 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d16107fb-6648-4083-ab38-dc0f21497f5a-sys\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.065497 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.065454 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.065497 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.065481 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d16107fb-6648-4083-ab38-dc0f21497f5a-root\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.126355 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.126310 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:11.167901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166211 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-wtmp\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.167901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166267 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l55pc\" (UniqueName: \"kubernetes.io/projected/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-kube-api-access-l55pc\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.167901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166300 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.167901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166329 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-textfile\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.167901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166362 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.167901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166387 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.167901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166415 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-wtmp\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.167901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166422 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n594p\" (UniqueName: \"kubernetes.io/projected/ffac38b9-3729-4e85-8128-ebf688f659fd-kube-api-access-n594p\") pod \"openshift-state-metrics-54cf96dfcf-xb8hz\" (UID: \"ffac38b9-3729-4e85-8128-ebf688f659fd\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:11.167901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166493 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d16107fb-6648-4083-ab38-dc0f21497f5a-sys\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.167901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166520 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.167901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166551 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d16107fb-6648-4083-ab38-dc0f21497f5a-root\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.167901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166601 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffac38b9-3729-4e85-8128-ebf688f659fd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-xb8hz\" (UID: \"ffac38b9-3729-4e85-8128-ebf688f659fd\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:11.167901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166623 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.168713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166670 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-accelerators-collector-config\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.168713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166699 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffac38b9-3729-4e85-8128-ebf688f659fd-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-xb8hz\" (UID: \"ffac38b9-3729-4e85-8128-ebf688f659fd\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:11.168713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166722 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-tls\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.168713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166749 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.168713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166797 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffac38b9-3729-4e85-8128-ebf688f659fd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-xb8hz\" (UID: \"ffac38b9-3729-4e85-8128-ebf688f659fd\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:11.168713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166839 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d16107fb-6648-4083-ab38-dc0f21497f5a-metrics-client-ca\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.168713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166872 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bxh6\" (UniqueName: \"kubernetes.io/projected/d16107fb-6648-4083-ab38-dc0f21497f5a-kube-api-access-6bxh6\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.168713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166884 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-textfile\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.168713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.166967 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d16107fb-6648-4083-ab38-dc0f21497f5a-sys\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.168713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.167278 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.168713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.167332 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d16107fb-6648-4083-ab38-dc0f21497f5a-root\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.168713 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:15:11.167407 2568 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 07 13:15:11.168713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.167428 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.169309 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:15:11.167463 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffac38b9-3729-4e85-8128-ebf688f659fd-openshift-state-metrics-tls podName:ffac38b9-3729-4e85-8128-ebf688f659fd nodeName:}" failed. No retries permitted until 2026-07-07 13:15:11.667445373 +0000 UTC m=+69.346006700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ffac38b9-3729-4e85-8128-ebf688f659fd-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-xb8hz" (UID: "ffac38b9-3729-4e85-8128-ebf688f659fd") : secret "openshift-state-metrics-tls" not found Jul 07 13:15:11.169309 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.167881 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.169309 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.169073 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-accelerators-collector-config\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.169805 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:15:11.169551 2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 13:15:11.169805 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:15:11.169624 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-tls podName:d16107fb-6648-4083-ab38-dc0f21497f5a nodeName:}" failed. No retries permitted until 2026-07-07 13:15:11.669607878 +0000 UTC m=+69.348169194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-tls") pod "node-exporter-7vrl2" (UID: "d16107fb-6648-4083-ab38-dc0f21497f5a") : secret "node-exporter-tls" not found Jul 07 13:15:11.169805 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.169654 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d16107fb-6648-4083-ab38-dc0f21497f5a-metrics-client-ca\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.169805 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.169699 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffac38b9-3729-4e85-8128-ebf688f659fd-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-xb8hz\" (UID: \"ffac38b9-3729-4e85-8128-ebf688f659fd\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:11.171417 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.171369 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.174199 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.174149 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.175905 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.174732 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.178502 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.178431 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffac38b9-3729-4e85-8128-ebf688f659fd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-xb8hz\" (UID: \"ffac38b9-3729-4e85-8128-ebf688f659fd\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:11.181297 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.181251 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n594p\" (UniqueName: \"kubernetes.io/projected/ffac38b9-3729-4e85-8128-ebf688f659fd-kube-api-access-n594p\") pod \"openshift-state-metrics-54cf96dfcf-xb8hz\" (UID: \"ffac38b9-3729-4e85-8128-ebf688f659fd\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:11.182362 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.182167 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l55pc\" (UniqueName: \"kubernetes.io/projected/0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0-kube-api-access-l55pc\") pod \"kube-state-metrics-7c9dbb8b7d-tmbbp\" (UID: \"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.183374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.183333 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bxh6\" (UniqueName: \"kubernetes.io/projected/d16107fb-6648-4083-ab38-dc0f21497f5a-kube-api-access-6bxh6\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.231825 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.231771 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:11.235416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.235387 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-25qdm" event={"ID":"faf07002-635a-4f2b-a452-be3e757a9d2f","Type":"ContainerStarted","Data":"11b2852047a87b4748ef3901824a548d38f068f452cf452ff20912220c3eb451"} Jul 07 13:15:11.237450 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.237380 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7cw9x" event={"ID":"6037cecb-d996-4bd8-a3d3-733a764f5163","Type":"ContainerStarted","Data":"9eef3f3c6400d66b797908d7c6aed577a431632d5b053d5fc3fd23290ffe91fb"} Jul 07 13:15:11.253048 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.253006 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" Jul 07 13:15:11.261737 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.260752 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-25qdm" podStartSLOduration=3.357017891 podStartE2EDuration="13.260730983s" totalImagesPullingTime="9.903713092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:14:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:11.259859235 +0000 UTC m=+68.938420567" watchObservedRunningTime="2026-07-07 13:15:11.260730983 +0000 UTC m=+68.939292318" Jul 07 13:15:11.334594 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.334552 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:11.434808 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.434696 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:11.541712 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.541617 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp"] Jul 07 13:15:11.547279 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:15:11.547241 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ea7c55e_4cdf_4cc5_8e6c_0da0af9ad5a0.slice/crio-b80d2124e6ee02187839c63c10a91c80bc282263a5c3871320199469f18f4dd8 WatchSource:0}: Error finding container b80d2124e6ee02187839c63c10a91c80bc282263a5c3871320199469f18f4dd8: Status 404 returned error can't find the container with id b80d2124e6ee02187839c63c10a91c80bc282263a5c3871320199469f18f4dd8 Jul 07 13:15:11.671816 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.671767 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffac38b9-3729-4e85-8128-ebf688f659fd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-xb8hz\" (UID: \"ffac38b9-3729-4e85-8128-ebf688f659fd\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:11.671962 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.671834 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-tls\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.675100 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.675064 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffac38b9-3729-4e85-8128-ebf688f659fd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-xb8hz\" (UID: \"ffac38b9-3729-4e85-8128-ebf688f659fd\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:11.675246 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.675208 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d16107fb-6648-4083-ab38-dc0f21497f5a-node-exporter-tls\") pod \"node-exporter-7vrl2\" (UID: \"d16107fb-6648-4083-ab38-dc0f21497f5a\") " pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.843830 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.843712 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" Jul 07 13:15:11.877349 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.877314 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7vrl2" Jul 07 13:15:11.952299 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.952258 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:11.994208 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:11.994182 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz"] Jul 07 13:15:11.996866 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:15:11.996835 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffac38b9_3729_4e85_8128_ebf688f659fd.slice/crio-0e396036f679a7005a037e5da1a2b49209a6bf19c7b0b39ef0a1923da01183db WatchSource:0}: Error finding container 0e396036f679a7005a037e5da1a2b49209a6bf19c7b0b39ef0a1923da01183db: Status 404 returned error can't find the container with id 0e396036f679a7005a037e5da1a2b49209a6bf19c7b0b39ef0a1923da01183db Jul 07 13:15:12.073871 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.073842 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:12.075331 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.075289 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 13:15:12.078507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.078487 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.081181 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.081162 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 13:15:12.081522 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.081503 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 13:15:12.082714 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.082630 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 13:15:12.082714 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.082709 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 13:15:12.083858 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.082914 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 13:15:12.083858 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.082931 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 13:15:12.083858 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.083198 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 13:15:12.083858 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.083227 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 13:15:12.083858 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.083555 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-nsz2b\"" Jul 07 13:15:12.083858 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.083602 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 13:15:12.099762 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.099625 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 13:15:12.176887 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.176758 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.176887 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.176810 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5spdv\" (UniqueName: \"kubernetes.io/projected/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-kube-api-access-5spdv\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.176887 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.176844 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.176887 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.176891 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.177364 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.176922 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-config-volume\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.177364 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.176947 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.177364 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.176977 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.177364 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.177003 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-config-out\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.177364 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.177034 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-web-config\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.177364 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.177144 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.177364 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.177182 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.177364 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.177219 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.177364 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.177276 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.250418 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.250319 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" event={"ID":"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0","Type":"ContainerStarted","Data":"b80d2124e6ee02187839c63c10a91c80bc282263a5c3871320199469f18f4dd8"} Jul 07 13:15:12.252714 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.252679 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6l86h" event={"ID":"f63deb5a-7f0d-4cca-a94a-3f3e0892379b","Type":"ContainerStarted","Data":"fada4227ce259b80cfccbe75045b468ba2a1bee4f27f5d1648d2ef51d8aab01e"} Jul 07 13:15:12.252885 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.252747 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6l86h" event={"ID":"f63deb5a-7f0d-4cca-a94a-3f3e0892379b","Type":"ContainerStarted","Data":"f75b00932376e3811d216118d7cf7d27ea704d36d59639756acc45e69782613a"} Jul 07 13:15:12.255532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.255451 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7vrl2" event={"ID":"d16107fb-6648-4083-ab38-dc0f21497f5a","Type":"ContainerStarted","Data":"c943915e0b87b1a31bc13602554b557be6c2ceddbd51e1c22ca45f8acf21b531"} Jul 07 13:15:12.259775 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.259519 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:12.259775 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.259632 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" event={"ID":"ffac38b9-3729-4e85-8128-ebf688f659fd","Type":"ContainerStarted","Data":"48c8282aa0da221caa87b927d101084e9e13dbd9e94689049ea0afad3ccc9d2d"} Jul 07 13:15:12.259775 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.259695 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" event={"ID":"ffac38b9-3729-4e85-8128-ebf688f659fd","Type":"ContainerStarted","Data":"779a51c344203e84d96e62a0ef26119e26f9cbdae8b2dca2b3c6306374affd7e"} Jul 07 13:15:12.259775 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.259729 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" event={"ID":"ffac38b9-3729-4e85-8128-ebf688f659fd","Type":"ContainerStarted","Data":"0e396036f679a7005a037e5da1a2b49209a6bf19c7b0b39ef0a1923da01183db"} Jul 07 13:15:12.278071 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.278036 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.278071 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.278075 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.278315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.278093 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.278315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.278125 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.278315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.278151 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5spdv\" (UniqueName: \"kubernetes.io/projected/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-kube-api-access-5spdv\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.278315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.278180 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.278315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.278201 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.278315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.278223 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-config-volume\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.278315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.278240 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.278315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.278255 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.278315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.278275 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-config-out\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.278315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.278291 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-web-config\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.278835 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.278337 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.278835 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.278798 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.279989 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.279653 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.279989 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.279706 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.281947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.281902 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.282881 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.282856 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.283851 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.283705 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-web-config\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.284892 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.284871 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.285756 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.285703 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.286720 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.286655 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6l86h" podStartSLOduration=66.751162703 podStartE2EDuration="1m9.286638479s" totalImagesPullingTime="2.535475776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:12.286434301 +0000 UTC m=+69.964995633" watchObservedRunningTime="2026-07-07 13:15:12.286638479 +0000 UTC m=+69.965199807" Jul 07 13:15:12.287135 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.287115 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.287223 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.287115 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.287338 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.287300 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-config-out\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.287754 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.287717 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-config-volume\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.295986 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.295939 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5spdv\" (UniqueName: \"kubernetes.io/projected/3e8df037-10ed-46f6-abd6-472d9ed3c1ef-kube-api-access-5spdv\") pod \"alertmanager-main-0\" (UID: \"3e8df037-10ed-46f6-abd6-472d9ed3c1ef\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.394726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.394552 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 13:15:12.476727 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.476658 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:12.572393 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.572274 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 13:15:12.575121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:12.575045 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:12.577740 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:15:12.577709 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e8df037_10ed_46f6_abd6_472d9ed3c1ef.slice/crio-11adf84f059c8010a6b408904c1e46bee9ea9cc3162c7c4c65589bedf145e7d1 WatchSource:0}: Error finding container 11adf84f059c8010a6b408904c1e46bee9ea9cc3162c7c4c65589bedf145e7d1: Status 404 returned error can't find the container with id 11adf84f059c8010a6b408904c1e46bee9ea9cc3162c7c4c65589bedf145e7d1 Jul 07 13:15:13.266647 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:13.266535 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e8df037-10ed-46f6-abd6-472d9ed3c1ef","Type":"ContainerStarted","Data":"11adf84f059c8010a6b408904c1e46bee9ea9cc3162c7c4c65589bedf145e7d1"} Jul 07 13:15:14.093598 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.093550 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:14.201632 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.201599 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qzrm2" Jul 07 13:15:14.211745 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.211714 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7566859584-k6vrm"] Jul 07 13:15:14.214680 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.214654 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.228053 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.228022 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 13:15:14.228196 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.228110 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 13:15:14.228196 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.228152 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 13:15:14.228196 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.228182 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 13:15:14.228335 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.228186 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 13:15:14.228335 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.228299 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-t6f2v\"" Jul 07 13:15:14.228640 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.228624 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-9hpqd1cikpnfe\"" Jul 07 13:15:14.277180 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.277149 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7566859584-k6vrm"] Jul 07 13:15:14.297182 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.297147 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.297340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.297191 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.297340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.297212 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-grpc-tls\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.297340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.297307 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.297543 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.297354 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7kqm\" (UniqueName: \"kubernetes.io/projected/33c98f55-320a-437c-b136-60180fc3ff4e-kube-api-access-j7kqm\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.297543 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.297440 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-tls\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.297543 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.297482 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.297543 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.297508 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33c98f55-320a-437c-b136-60180fc3ff4e-metrics-client-ca\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.313632 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.313592 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:14.398099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.398018 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.398099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.398062 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33c98f55-320a-437c-b136-60180fc3ff4e-metrics-client-ca\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.398315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.398103 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.398315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.398134 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.398315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.398154 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-grpc-tls\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.398315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.398194 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.398315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.398231 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7kqm\" (UniqueName: \"kubernetes.io/projected/33c98f55-320a-437c-b136-60180fc3ff4e-kube-api-access-j7kqm\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.398315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.398268 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-tls\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.399031 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.398978 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33c98f55-320a-437c-b136-60180fc3ff4e-metrics-client-ca\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.401268 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.401235 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.401385 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.401334 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.401502 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.401478 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-grpc-tls\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.401667 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.401617 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.401667 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.401636 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-tls\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.401797 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.401748 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/33c98f55-320a-437c-b136-60180fc3ff4e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.413270 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.413247 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7kqm\" (UniqueName: \"kubernetes.io/projected/33c98f55-320a-437c-b136-60180fc3ff4e-kube-api-access-j7kqm\") pod \"thanos-querier-7566859584-k6vrm\" (UID: \"33c98f55-320a-437c-b136-60180fc3ff4e\") " pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.442889 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.442857 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:14.525323 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.525279 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:14.567456 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:14.567424 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:15.292965 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.292937 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-c8557b76-7gc7w"] Jul 07 13:15:15.296476 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.296454 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.303979 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.303959 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 07 13:15:15.303979 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.303968 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 07 13:15:15.304464 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.304443 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 13:15:15.304558 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.304504 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 07 13:15:15.305099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.305084 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-6v9b4\"" Jul 07 13:15:15.305099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.305091 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6jlefkvteanq8\"" Jul 07 13:15:15.313218 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.313191 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c8557b76-7gc7w"] Jul 07 13:15:15.408521 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.408108 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/79e96c1a-298b-465b-9709-c203521ae225-client-ca-bundle\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.408521 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.408172 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/79e96c1a-298b-465b-9709-c203521ae225-metrics-server-audit-profiles\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.408521 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.408242 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79e96c1a-298b-465b-9709-c203521ae225-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.408521 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.408301 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/79e96c1a-298b-465b-9709-c203521ae225-audit-log\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.408521 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.408338 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgfm4\" (UniqueName: \"kubernetes.io/projected/79e96c1a-298b-465b-9709-c203521ae225-kube-api-access-wgfm4\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.408521 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.408377 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/79e96c1a-298b-465b-9709-c203521ae225-secret-metrics-server-client-certs\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.408521 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.408428 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/79e96c1a-298b-465b-9709-c203521ae225-secret-metrics-server-tls\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.509597 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.509545 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/79e96c1a-298b-465b-9709-c203521ae225-audit-log\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.509782 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.509617 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgfm4\" (UniqueName: \"kubernetes.io/projected/79e96c1a-298b-465b-9709-c203521ae225-kube-api-access-wgfm4\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.509782 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.509654 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/79e96c1a-298b-465b-9709-c203521ae225-secret-metrics-server-client-certs\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.509782 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.509691 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/79e96c1a-298b-465b-9709-c203521ae225-secret-metrics-server-tls\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.509782 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.509729 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/79e96c1a-298b-465b-9709-c203521ae225-client-ca-bundle\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.509782 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.509751 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/79e96c1a-298b-465b-9709-c203521ae225-metrics-server-audit-profiles\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.509982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.509790 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79e96c1a-298b-465b-9709-c203521ae225-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.510035 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.509981 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/79e96c1a-298b-465b-9709-c203521ae225-audit-log\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.510648 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.510596 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79e96c1a-298b-465b-9709-c203521ae225-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.510964 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.510917 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/79e96c1a-298b-465b-9709-c203521ae225-metrics-server-audit-profiles\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.512798 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.512771 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/79e96c1a-298b-465b-9709-c203521ae225-secret-metrics-server-tls\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.512924 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.512904 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/79e96c1a-298b-465b-9709-c203521ae225-client-ca-bundle\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.512982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.512920 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/79e96c1a-298b-465b-9709-c203521ae225-secret-metrics-server-client-certs\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.519326 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.519289 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgfm4\" (UniqueName: \"kubernetes.io/projected/79e96c1a-298b-465b-9709-c203521ae225-kube-api-access-wgfm4\") pod \"metrics-server-c8557b76-7gc7w\" (UID: \"79e96c1a-298b-465b-9709-c203521ae225\") " pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.606855 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.606762 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:15.638813 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.638759 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:15.661738 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.661700 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n"] Jul 07 13:15:15.665667 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.665642 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bdb7f747d-r5kkp"] Jul 07 13:15:15.665972 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.665943 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" Jul 07 13:15:15.667934 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.667915 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.669202 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.669183 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 07 13:15:15.669304 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.669215 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-d5dkd\"" Jul 07 13:15:15.675327 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.675305 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n"] Jul 07 13:15:15.690002 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.689980 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bdb7f747d-r5kkp"] Jul 07 13:15:15.728409 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.726836 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:15.812564 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.812530 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-oauth-config\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.812727 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.812586 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-serving-cert\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.812727 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.812619 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw7h5\" (UniqueName: \"kubernetes.io/projected/54d7b121-70f9-41b2-8a1c-5e5ff2f37533-kube-api-access-lw7h5\") pod \"monitoring-plugin-d5dcff7cf-fdn9n\" (UID: \"54d7b121-70f9-41b2-8a1c-5e5ff2f37533\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" Jul 07 13:15:15.812727 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.812648 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-config\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.812727 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.812713 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-trusted-ca-bundle\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.812924 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.812765 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-oauth-serving-cert\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.812924 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.812801 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-service-ca\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.812924 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.812826 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d7b121-70f9-41b2-8a1c-5e5ff2f37533-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-fdn9n\" (UID: \"54d7b121-70f9-41b2-8a1c-5e5ff2f37533\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" Jul 07 13:15:15.812924 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.812875 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v55xx\" (UniqueName: \"kubernetes.io/projected/a60c7784-bf24-4c6b-8d17-d704afb646c0-kube-api-access-v55xx\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.818987 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.818957 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:15.909926 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.909833 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:15.913884 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.913859 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-oauth-config\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.913998 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.913894 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-serving-cert\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.913998 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.913916 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw7h5\" (UniqueName: \"kubernetes.io/projected/54d7b121-70f9-41b2-8a1c-5e5ff2f37533-kube-api-access-lw7h5\") pod \"monitoring-plugin-d5dcff7cf-fdn9n\" (UID: \"54d7b121-70f9-41b2-8a1c-5e5ff2f37533\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" Jul 07 13:15:15.913998 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.913959 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-config\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.914145 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.914002 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-trusted-ca-bundle\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.914145 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.914032 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-oauth-serving-cert\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.914145 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.914064 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-service-ca\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.914145 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.914085 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d7b121-70f9-41b2-8a1c-5e5ff2f37533-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-fdn9n\" (UID: \"54d7b121-70f9-41b2-8a1c-5e5ff2f37533\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" Jul 07 13:15:15.914145 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.914110 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v55xx\" (UniqueName: \"kubernetes.io/projected/a60c7784-bf24-4c6b-8d17-d704afb646c0-kube-api-access-v55xx\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.914553 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:15:15.914466 2568 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 07 13:15:15.914553 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:15:15.914545 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54d7b121-70f9-41b2-8a1c-5e5ff2f37533-monitoring-plugin-cert podName:54d7b121-70f9-41b2-8a1c-5e5ff2f37533 nodeName:}" failed. No retries permitted until 2026-07-07 13:15:16.414524942 +0000 UTC m=+74.093086257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/54d7b121-70f9-41b2-8a1c-5e5ff2f37533-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-fdn9n" (UID: "54d7b121-70f9-41b2-8a1c-5e5ff2f37533") : secret "monitoring-plugin-cert" not found Jul 07 13:15:15.914908 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.914850 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-oauth-serving-cert\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.914908 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.914888 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-config\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.915072 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.914999 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-trusted-ca-bundle\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.915507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.915460 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-service-ca\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.916866 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.916844 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-oauth-config\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.916971 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.916883 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-serving-cert\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.926646 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.926619 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw7h5\" (UniqueName: \"kubernetes.io/projected/54d7b121-70f9-41b2-8a1c-5e5ff2f37533-kube-api-access-lw7h5\") pod \"monitoring-plugin-d5dcff7cf-fdn9n\" (UID: \"54d7b121-70f9-41b2-8a1c-5e5ff2f37533\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" Jul 07 13:15:15.926763 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.926736 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v55xx\" (UniqueName: \"kubernetes.io/projected/a60c7784-bf24-4c6b-8d17-d704afb646c0-kube-api-access-v55xx\") pod \"console-bdb7f747d-r5kkp\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:15.986125 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:15.986089 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:16.092214 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:16.092177 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:16.171505 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:16.171476 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:16.304207 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:16.304110 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:16.410099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:16.409979 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:16.420647 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:16.419970 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d7b121-70f9-41b2-8a1c-5e5ff2f37533-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-fdn9n\" (UID: \"54d7b121-70f9-41b2-8a1c-5e5ff2f37533\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" Jul 07 13:15:16.420647 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:15:16.420144 2568 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 07 13:15:16.420647 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:15:16.420203 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54d7b121-70f9-41b2-8a1c-5e5ff2f37533-monitoring-plugin-cert podName:54d7b121-70f9-41b2-8a1c-5e5ff2f37533 nodeName:}" failed. No retries permitted until 2026-07-07 13:15:17.420184558 +0000 UTC m=+75.098745890 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/54d7b121-70f9-41b2-8a1c-5e5ff2f37533-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-fdn9n" (UID: "54d7b121-70f9-41b2-8a1c-5e5ff2f37533") : secret "monitoring-plugin-cert" not found Jul 07 13:15:16.503207 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:16.502798 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7566859584-k6vrm"] Jul 07 13:15:16.503207 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:15:16.503021 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c98f55_320a_437c_b136_60180fc3ff4e.slice/crio-86be07b310e0cfe53ee5917b7e578eb5471454906d23e82ab09cb11bb5b44fcc WatchSource:0}: Error finding container 86be07b310e0cfe53ee5917b7e578eb5471454906d23e82ab09cb11bb5b44fcc: Status 404 returned error can't find the container with id 86be07b310e0cfe53ee5917b7e578eb5471454906d23e82ab09cb11bb5b44fcc Jul 07 13:15:16.504196 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:16.504118 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c8557b76-7gc7w"] Jul 07 13:15:16.529886 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:16.524527 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bdb7f747d-r5kkp"] Jul 07 13:15:16.825711 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:16.825680 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:17.291293 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.291255 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" event={"ID":"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0","Type":"ContainerStarted","Data":"fc422134ccdeef8014e34e9d877225b6d628718217af1129ab6b143aad236c6e"} Jul 07 13:15:17.291504 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.291322 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" event={"ID":"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0","Type":"ContainerStarted","Data":"cd0aeada22cc2b747d98e182fadc99b7095bdace280050a8a7ddc96269f70991"} Jul 07 13:15:17.291504 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.291363 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" event={"ID":"0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0","Type":"ContainerStarted","Data":"f07402725ce69432ccfdf74e32a09af2e1cbe4c1daa74fda2cb538b1ab5fb025"} Jul 07 13:15:17.292977 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.292952 2568 generic.go:356] "Generic (PLEG): container finished" podID="3e8df037-10ed-46f6-abd6-472d9ed3c1ef" containerID="e7a29a090ad8a9a4332042de7ab186b9a3154f94b1555647e234b50e1e922256" exitCode=0 Jul 07 13:15:17.293117 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.293043 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e8df037-10ed-46f6-abd6-472d9ed3c1ef","Type":"ContainerDied","Data":"e7a29a090ad8a9a4332042de7ab186b9a3154f94b1555647e234b50e1e922256"} Jul 07 13:15:17.294545 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.294520 2568 generic.go:356] "Generic (PLEG): container finished" podID="d16107fb-6648-4083-ab38-dc0f21497f5a" containerID="1426b2ba7d4bdaad7d51590fd2f9385cb32c808a9ed799823d0e6b3c3426840b" exitCode=0 Jul 07 13:15:17.294670 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.294619 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7vrl2" event={"ID":"d16107fb-6648-4083-ab38-dc0f21497f5a","Type":"ContainerDied","Data":"1426b2ba7d4bdaad7d51590fd2f9385cb32c808a9ed799823d0e6b3c3426840b"} Jul 07 13:15:17.298929 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.298903 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" event={"ID":"ffac38b9-3729-4e85-8128-ebf688f659fd","Type":"ContainerStarted","Data":"5af6e85cdf73934a7279ef1e02f564ce88dee2c4300793ea42af97cb28cdb7dd"} Jul 07 13:15:17.301253 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.301147 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j2rqx" event={"ID":"730e3c49-83b7-4a1d-9b7c-d5621955f6df","Type":"ContainerStarted","Data":"986fef6acfdde799b3b5ad7ad7adf5a46ff68e62e2cdcfb419d47fe70a427572"} Jul 07 13:15:17.303437 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.303411 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7cw9x" event={"ID":"6037cecb-d996-4bd8-a3d3-733a764f5163","Type":"ContainerStarted","Data":"12a92aacdedd38b2412036b7c7897b9fd8d44c868632e14fad4bd37e047aa92b"} Jul 07 13:15:17.303547 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.303472 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:15:17.304717 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.304694 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" event={"ID":"79e96c1a-298b-465b-9709-c203521ae225","Type":"ContainerStarted","Data":"01f6d2953e140791330bb5b50b63b82bf640eeb2e022af574cec76fa67f67b95"} Jul 07 13:15:17.306512 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.306490 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bdb7f747d-r5kkp" event={"ID":"a60c7784-bf24-4c6b-8d17-d704afb646c0","Type":"ContainerStarted","Data":"9bef97df1559feeef36b2bdbd1a3f7ba1529720c21b6a6edd1f5cf8692604f90"} Jul 07 13:15:17.306609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.306528 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bdb7f747d-r5kkp" event={"ID":"a60c7784-bf24-4c6b-8d17-d704afb646c0","Type":"ContainerStarted","Data":"8126cd7801da395ce78046318ff6cadb3168edcf7a1b83018d2f10f18bb88a33"} Jul 07 13:15:17.308037 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.308019 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" event={"ID":"33c98f55-320a-437c-b136-60180fc3ff4e","Type":"ContainerStarted","Data":"86be07b310e0cfe53ee5917b7e578eb5471454906d23e82ab09cb11bb5b44fcc"} Jul 07 13:15:17.321330 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.321281 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tmbbp" podStartSLOduration=2.642444291 podStartE2EDuration="7.321263544s" totalImagesPullingTime="4.678819253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:17.320000687 +0000 UTC m=+74.998562021" watchObservedRunningTime="2026-07-07 13:15:17.321263544 +0000 UTC m=+74.999824874" Jul 07 13:15:17.347007 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.346952 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xb8hz" podStartSLOduration=3.262224093 podStartE2EDuration="7.346930234s" totalImagesPullingTime="4.084706141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:17.345993142 +0000 UTC m=+75.024554473" watchObservedRunningTime="2026-07-07 13:15:17.346930234 +0000 UTC m=+75.025491567" Jul 07 13:15:17.374139 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.373819 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7cw9x" podStartSLOduration=68.427513932 podStartE2EDuration="1m14.373796899s" totalImagesPullingTime="5.946282967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:17.373434241 +0000 UTC m=+75.051995574" watchObservedRunningTime="2026-07-07 13:15:17.373796899 +0000 UTC m=+75.052358231" Jul 07 13:15:17.394820 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.394768 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j2rqx" podStartSLOduration=66.977475305 podStartE2EDuration="1m14.394750064s" totalImagesPullingTime="7.417274759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:17.391859167 +0000 UTC m=+75.070420499" watchObservedRunningTime="2026-07-07 13:15:17.394750064 +0000 UTC m=+75.073311394" Jul 07 13:15:17.395260 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.395232 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:17.397047 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.396938 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 13:15:17.402424 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.402408 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.407073 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.406736 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 13:15:17.407073 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.406770 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 13:15:17.409430 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.409409 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 13:15:17.409538 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.409500 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 13:15:17.409538 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.409526 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 13:15:17.409717 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.409700 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 13:15:17.409808 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.409770 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 13:15:17.409873 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.409822 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 13:15:17.409873 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.409866 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 13:15:17.409978 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.409875 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 13:15:17.410034 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.409989 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 13:15:17.410217 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.410200 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7f53qsod7au2s\"" Jul 07 13:15:17.410417 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.410398 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 13:15:17.410490 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.410469 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hml9k\"" Jul 07 13:15:17.410820 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.410800 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 13:15:17.411106 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.411089 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 13:15:17.430750 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.430687 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 13:15:17.437542 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.435013 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d7b121-70f9-41b2-8a1c-5e5ff2f37533-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-fdn9n\" (UID: \"54d7b121-70f9-41b2-8a1c-5e5ff2f37533\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" Jul 07 13:15:17.441243 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.441223 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d7b121-70f9-41b2-8a1c-5e5ff2f37533-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-fdn9n\" (UID: \"54d7b121-70f9-41b2-8a1c-5e5ff2f37533\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" Jul 07 13:15:17.479272 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.479224 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" Jul 07 13:15:17.486136 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.485016 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bdb7f747d-r5kkp" podStartSLOduration=2.484981065 podStartE2EDuration="2.484981065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:17.482984509 +0000 UTC m=+75.161545840" watchObservedRunningTime="2026-07-07 13:15:17.484981065 +0000 UTC m=+75.163542379" Jul 07 13:15:17.536959 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.536888 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.536959 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.536940 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537211 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.536981 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537211 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537035 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537211 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537067 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537211 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537104 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcks9\" (UniqueName: \"kubernetes.io/projected/505e1ce2-f48e-463b-9eae-b6ae88f14234-kube-api-access-zcks9\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537211 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537134 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537211 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537174 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537500 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537239 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537500 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537285 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/505e1ce2-f48e-463b-9eae-b6ae88f14234-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537500 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537313 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537500 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537342 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-config\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537500 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537370 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537500 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537395 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537500 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537410 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-web-config\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537500 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537442 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537500 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537467 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537500 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537500 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537998 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537531 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/505e1ce2-f48e-463b-9eae-b6ae88f14234-config-out\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.537998 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.537547 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.639229 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.638860 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.639229 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.638912 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.639229 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.638952 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/505e1ce2-f48e-463b-9eae-b6ae88f14234-config-out\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.639229 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.638979 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.639229 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.639013 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.639229 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.639042 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.639845 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.639740 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.639845 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.639825 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.639986 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.639857 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.639986 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.639899 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcks9\" (UniqueName: \"kubernetes.io/projected/505e1ce2-f48e-463b-9eae-b6ae88f14234-kube-api-access-zcks9\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.639986 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.639926 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.639986 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.639964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.640205 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.640008 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.640205 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.640053 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/505e1ce2-f48e-463b-9eae-b6ae88f14234-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.640205 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.640077 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.640205 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.640109 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-config\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.640205 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.640143 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.640205 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.640171 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.640205 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.640187 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.640205 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.640200 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-web-config\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.640610 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.640254 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.641293 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.640690 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.641534 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.641508 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.642006 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.641983 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.642863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.642839 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.643461 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.643437 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.644737 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.644293 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.647839 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.647658 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n"] Jul 07 13:15:17.650164 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.650117 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/505e1ce2-f48e-463b-9eae-b6ae88f14234-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.650600 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.650410 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/505e1ce2-f48e-463b-9eae-b6ae88f14234-config-out\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.650918 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.650808 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.650918 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.650859 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-web-config\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.650918 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.650882 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.651381 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.651115 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.652154 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.652111 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.656292 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:15:17.656260 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d7b121_70f9_41b2_8a1c_5e5ff2f37533.slice/crio-23277fdc68a1abef43e7ba8041227d6788bb44b91e28b7420cc70ff8476beaf6 WatchSource:0}: Error finding container 23277fdc68a1abef43e7ba8041227d6788bb44b91e28b7420cc70ff8476beaf6: Status 404 returned error can't find the container with id 23277fdc68a1abef43e7ba8041227d6788bb44b91e28b7420cc70ff8476beaf6 Jul 07 13:15:17.658102 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.657564 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.658102 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.657584 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-config\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.658421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.658271 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/505e1ce2-f48e-463b-9eae-b6ae88f14234-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.658421 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.658387 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.659569 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.658912 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/505e1ce2-f48e-463b-9eae-b6ae88f14234-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.659569 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.659259 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcks9\" (UniqueName: \"kubernetes.io/projected/505e1ce2-f48e-463b-9eae-b6ae88f14234-kube-api-access-zcks9\") pod \"prometheus-k8s-0\" (UID: \"505e1ce2-f48e-463b-9eae-b6ae88f14234\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.726493 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.726381 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:17.954824 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:17.954716 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 13:15:18.306750 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:18.306703 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:18.317115 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:18.317076 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7vrl2" event={"ID":"d16107fb-6648-4083-ab38-dc0f21497f5a","Type":"ContainerStarted","Data":"992639cf4798a563d57f49e6c0e7c5089764ab3071add432a628f89edab6e88b"} Jul 07 13:15:18.317308 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:18.317165 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7vrl2" event={"ID":"d16107fb-6648-4083-ab38-dc0f21497f5a","Type":"ContainerStarted","Data":"9c9364055256cd1ca99f572f600d29f94ba1ed094bbe7278f10648b8bb6238ac"} Jul 07 13:15:18.319125 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:18.319047 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"505e1ce2-f48e-463b-9eae-b6ae88f14234","Type":"ContainerStarted","Data":"4335193a826e75e97c27f66f68fa1353203ec7cbf31460c821cf410cf73f5060"} Jul 07 13:15:18.320799 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:18.320765 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" event={"ID":"54d7b121-70f9-41b2-8a1c-5e5ff2f37533","Type":"ContainerStarted","Data":"23277fdc68a1abef43e7ba8041227d6788bb44b91e28b7420cc70ff8476beaf6"} Jul 07 13:15:18.341872 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:18.341535 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7vrl2" podStartSLOduration=3.862494865 podStartE2EDuration="8.341515096s" totalImagesPullingTime="4.342137895s" totalInitContainerRuntime="136.882336ms" isStatefulPod=true podCreationTimestamp="2026-07-07 13:15:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:18.337708094 +0000 UTC m=+76.016269426" watchObservedRunningTime="2026-07-07 13:15:18.341515096 +0000 UTC m=+76.020076428" Jul 07 13:15:18.510235 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:18.510205 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:19.332277 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:19.332235 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" event={"ID":"79e96c1a-298b-465b-9709-c203521ae225","Type":"ContainerStarted","Data":"8d1157121febada156f5e647cf02a01bcf2e9aa89b49c9f0fdc6b73fd6723ccd"} Jul 07 13:15:19.334354 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:19.334324 2568 generic.go:356] "Generic (PLEG): container finished" podID="505e1ce2-f48e-463b-9eae-b6ae88f14234" containerID="03ce31a20f6655474beb1b423f1b42d48f3f32355ef69f15ede25123aa652852" exitCode=0 Jul 07 13:15:19.334480 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:19.334407 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"505e1ce2-f48e-463b-9eae-b6ae88f14234","Type":"ContainerDied","Data":"03ce31a20f6655474beb1b423f1b42d48f3f32355ef69f15ede25123aa652852"} Jul 07 13:15:19.359778 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:19.359700 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" podStartSLOduration=2.35579809 podStartE2EDuration="4.35967668s" totalImagesPullingTime="2.00387859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:19.358166348 +0000 UTC m=+77.036727723" watchObservedRunningTime="2026-07-07 13:15:19.35967668 +0000 UTC m=+77.038238044" Jul 07 13:15:21.350050 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:21.350006 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e8df037-10ed-46f6-abd6-472d9ed3c1ef","Type":"ContainerStarted","Data":"2d6243c5e15bdacf9be1d8b35c57639189a971c5ac916bfef7c3dfe91e89033c"} Jul 07 13:15:21.350510 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:21.350098 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e8df037-10ed-46f6-abd6-472d9ed3c1ef","Type":"ContainerStarted","Data":"da532077ea8ae1aa434cb0f657f4113a9d53d70b4ea9d44bcc9456895d9af971"} Jul 07 13:15:21.350510 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:21.350138 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e8df037-10ed-46f6-abd6-472d9ed3c1ef","Type":"ContainerStarted","Data":"883c87c9d133f8dfa6604513868b68fe28847d755f0789e9a5c5b62f1f6dd290"} Jul 07 13:15:21.350510 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:21.350176 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e8df037-10ed-46f6-abd6-472d9ed3c1ef","Type":"ContainerStarted","Data":"d9b7c36591b8e5a14ba0c47f0cf593468e549a9119d0aad259da1e7c2f72270c"} Jul 07 13:15:21.350510 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:21.350200 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e8df037-10ed-46f6-abd6-472d9ed3c1ef","Type":"ContainerStarted","Data":"9e0f6ff791a67ca03bee9cb93935350a13c45983de66159baba9b7eff469e13f"} Jul 07 13:15:21.352740 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:21.352704 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" event={"ID":"33c98f55-320a-437c-b136-60180fc3ff4e","Type":"ContainerStarted","Data":"3097df70dbaad29903675a37b50f1a7d87520c3ae0bf842ad187473d6f1878ae"} Jul 07 13:15:21.352854 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:21.352779 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" event={"ID":"33c98f55-320a-437c-b136-60180fc3ff4e","Type":"ContainerStarted","Data":"b93f83608a0e161fcc0099a3cdd95cde27dbac266ea94e47d494cd1e2d1b0fb9"} Jul 07 13:15:21.352854 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:21.352820 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" event={"ID":"33c98f55-320a-437c-b136-60180fc3ff4e","Type":"ContainerStarted","Data":"65f5ff1d9776bf2b1f931d4e7c97bedebf00a8949ce0cb2229865db5c01cdd44"} Jul 07 13:15:21.354767 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:21.354736 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" event={"ID":"54d7b121-70f9-41b2-8a1c-5e5ff2f37533","Type":"ContainerStarted","Data":"0071a8f70129464f529811090d467362dc449dd7c649f67ae8dec54b6366d56e"} Jul 07 13:15:21.354978 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:21.354957 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" Jul 07 13:15:21.360913 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:21.360894 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" Jul 07 13:15:21.376132 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:21.375943 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-fdn9n" podStartSLOduration=3.614226802 podStartE2EDuration="6.375924245s" totalImagesPullingTime="2.761697443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:21.373770306 +0000 UTC m=+79.052331639" watchObservedRunningTime="2026-07-07 13:15:21.375924245 +0000 UTC m=+79.054485579" Jul 07 13:15:22.363452 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:22.363405 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" event={"ID":"33c98f55-320a-437c-b136-60180fc3ff4e","Type":"ContainerStarted","Data":"07ef1448100e23f22c9e86842b68773255f3124f24511b348401fbe5b7b4410b"} Jul 07 13:15:22.363973 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:22.363521 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" event={"ID":"33c98f55-320a-437c-b136-60180fc3ff4e","Type":"ContainerStarted","Data":"8dcde24f6fe1a2e792df7ba69055a154ad4d0b70fc9ed4153c761584349118dd"} Jul 07 13:15:22.363973 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:22.363558 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" event={"ID":"33c98f55-320a-437c-b136-60180fc3ff4e","Type":"ContainerStarted","Data":"1d513aab526278c20775e892b39fd8d54907c00d365c0006c92fee8acfe1eec3"} Jul 07 13:15:22.363973 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:22.363635 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:22.367416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:22.367380 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e8df037-10ed-46f6-abd6-472d9ed3c1ef","Type":"ContainerStarted","Data":"e0dd1005843b07eadc1fcff9ae0290b6eb57994333adba4f14207f86443cd8cb"} Jul 07 13:15:22.462512 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:22.462463 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.81914022 podStartE2EDuration="10.462445559s" totalImagesPullingTime="7.617230019s" totalInitContainerRuntime="26.07532ms" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:22.4603224 +0000 UTC m=+80.138883735" watchObservedRunningTime="2026-07-07 13:15:22.462445559 +0000 UTC m=+80.141006888" Jul 07 13:15:22.462835 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:22.462810 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" podStartSLOduration=3.615376006 podStartE2EDuration="8.462800866s" totalImagesPullingTime="4.84742486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:22.422011383 +0000 UTC m=+80.100572715" watchObservedRunningTime="2026-07-07 13:15:22.462800866 +0000 UTC m=+80.141362197" Jul 07 13:15:23.399515 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:23.399492 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:23.501902 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:23.501870 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:24.378462 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:24.378327 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"505e1ce2-f48e-463b-9eae-b6ae88f14234","Type":"ContainerStarted","Data":"566db266e037e04fe46d76eb9d5990888b008226448324013f59c45d3e2df077"} Jul 07 13:15:24.378462 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:24.378416 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"505e1ce2-f48e-463b-9eae-b6ae88f14234","Type":"ContainerStarted","Data":"22fe9d87e8dd619dab7265402666cb708466b01733c340cfaed0ce723d7546f1"} Jul 07 13:15:24.378462 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:24.378457 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"505e1ce2-f48e-463b-9eae-b6ae88f14234","Type":"ContainerStarted","Data":"b90047c6aa4c07d5dae106d3e4ce8b350440ab6f30882ec97d92f122cce587ee"} Jul 07 13:15:24.378754 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:24.378485 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"505e1ce2-f48e-463b-9eae-b6ae88f14234","Type":"ContainerStarted","Data":"883cc8dcd9e7f2ad49f70278d4dd4d0938792de600f9ec86c67925ff527194a6"} Jul 07 13:15:24.378754 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:24.378507 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"505e1ce2-f48e-463b-9eae-b6ae88f14234","Type":"ContainerStarted","Data":"97ee97a3af75468363e3634873bb2766ea1c396fd02c0ed5c3c812c95c0818ab"} Jul 07 13:15:24.378754 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:24.378545 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"505e1ce2-f48e-463b-9eae-b6ae88f14234","Type":"ContainerStarted","Data":"dcaa121aced0f82803a493996fa1ee39d4319d710a345147af304541679f80d2"} Jul 07 13:15:24.412569 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:24.412506 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.401147357 podStartE2EDuration="7.412484365s" totalImagesPullingTime="3.992290152s" totalInitContainerRuntime="19.046856ms" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:24.410779146 +0000 UTC m=+82.089340478" watchObservedRunningTime="2026-07-07 13:15:24.412484365 +0000 UTC m=+82.091045698" Jul 07 13:15:25.986471 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:25.986435 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:25.986981 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:25.986526 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:25.991102 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:25.991084 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:26.390897 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:26.390809 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:15:26.442399 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:26.442360 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-866f955bdc-g8wjb"] Jul 07 13:15:26.568450 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:26.568421 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:26.674265 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:26.674236 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:27.727256 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:27.727217 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:15:28.377164 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:28.377139 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7566859584-k6vrm" Jul 07 13:15:32.418934 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:32.418895 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:32.495244 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:32.495208 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:35.607333 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:35.607290 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:35.607859 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:35.607387 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:36.284226 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:36.284192 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:36.364048 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:36.363996 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:36.807871 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:36.807819 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9zqg5_627d53fa-7581-4f07-94ee-78932d124b75/serve-healthcheck-canary/0.log" Jul 07 13:15:39.297919 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:39.297864 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:39.436994 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:39.436956 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:39.716482 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:39.716449 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:39.866247 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:39.866210 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:39.982125 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:39.982026 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:47.374281 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:47.374231 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:47.450412 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:47.450369 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:48.325230 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:48.324390 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7cw9x" Jul 07 13:15:51.463881 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:51.463804 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-866f955bdc-g8wjb" podUID="3b69addc-9aea-42aa-a333-db21b736ba26" containerName="console" containerID="cri-o://04725e0c3c4c582f2faeee8fe52c9ad2ff241954966baacbcb9d2d5a0acd9f83" gracePeriod=15 Jul 07 13:15:52.222895 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.222863 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:15:52.259462 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.259430 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-console-config\") pod \"3b69addc-9aea-42aa-a333-db21b736ba26\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " Jul 07 13:15:52.259655 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.259493 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b69addc-9aea-42aa-a333-db21b736ba26-console-oauth-config\") pod \"3b69addc-9aea-42aa-a333-db21b736ba26\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " Jul 07 13:15:52.259655 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.259530 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fptbd\" (UniqueName: \"kubernetes.io/projected/3b69addc-9aea-42aa-a333-db21b736ba26-kube-api-access-fptbd\") pod \"3b69addc-9aea-42aa-a333-db21b736ba26\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " Jul 07 13:15:52.259655 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.259589 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-service-ca\") pod \"3b69addc-9aea-42aa-a333-db21b736ba26\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " Jul 07 13:15:52.259655 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.259622 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-trusted-ca-bundle\") pod \"3b69addc-9aea-42aa-a333-db21b736ba26\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " Jul 07 13:15:52.259655 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.259643 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b69addc-9aea-42aa-a333-db21b736ba26-console-serving-cert\") pod \"3b69addc-9aea-42aa-a333-db21b736ba26\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " Jul 07 13:15:52.259923 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.259665 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-oauth-serving-cert\") pod \"3b69addc-9aea-42aa-a333-db21b736ba26\" (UID: \"3b69addc-9aea-42aa-a333-db21b736ba26\") " Jul 07 13:15:52.260097 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.260045 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-console-config" (OuterVolumeSpecName: "console-config") pod "3b69addc-9aea-42aa-a333-db21b736ba26" (UID: "3b69addc-9aea-42aa-a333-db21b736ba26"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:52.260229 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.260187 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3b69addc-9aea-42aa-a333-db21b736ba26" (UID: "3b69addc-9aea-42aa-a333-db21b736ba26"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:52.260291 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.260218 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3b69addc-9aea-42aa-a333-db21b736ba26" (UID: "3b69addc-9aea-42aa-a333-db21b736ba26"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:52.260372 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.260343 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-service-ca" (OuterVolumeSpecName: "service-ca") pod "3b69addc-9aea-42aa-a333-db21b736ba26" (UID: "3b69addc-9aea-42aa-a333-db21b736ba26"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:52.262004 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.261980 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b69addc-9aea-42aa-a333-db21b736ba26-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3b69addc-9aea-42aa-a333-db21b736ba26" (UID: "3b69addc-9aea-42aa-a333-db21b736ba26"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:15:52.262004 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.261984 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b69addc-9aea-42aa-a333-db21b736ba26-kube-api-access-fptbd" (OuterVolumeSpecName: "kube-api-access-fptbd") pod "3b69addc-9aea-42aa-a333-db21b736ba26" (UID: "3b69addc-9aea-42aa-a333-db21b736ba26"). InnerVolumeSpecName "kube-api-access-fptbd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:52.262130 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.262037 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b69addc-9aea-42aa-a333-db21b736ba26-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3b69addc-9aea-42aa-a333-db21b736ba26" (UID: "3b69addc-9aea-42aa-a333-db21b736ba26"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:15:52.360557 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.360487 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-service-ca\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:15:52.360557 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.360512 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-trusted-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:15:52.360557 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.360523 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b69addc-9aea-42aa-a333-db21b736ba26-console-serving-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:15:52.360557 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.360532 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-oauth-serving-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:15:52.360557 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.360543 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b69addc-9aea-42aa-a333-db21b736ba26-console-config\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:15:52.360557 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.360551 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b69addc-9aea-42aa-a333-db21b736ba26-console-oauth-config\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:15:52.360557 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.360559 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fptbd\" (UniqueName: \"kubernetes.io/projected/3b69addc-9aea-42aa-a333-db21b736ba26-kube-api-access-fptbd\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:15:52.500444 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.500412 2568 generic.go:356] "Generic (PLEG): container finished" podID="3b69addc-9aea-42aa-a333-db21b736ba26" containerID="04725e0c3c4c582f2faeee8fe52c9ad2ff241954966baacbcb9d2d5a0acd9f83" exitCode=0 Jul 07 13:15:52.500839 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.500459 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-866f955bdc-g8wjb" Jul 07 13:15:52.500839 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.500489 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-866f955bdc-g8wjb" event={"ID":"3b69addc-9aea-42aa-a333-db21b736ba26","Type":"ContainerDied","Data":"04725e0c3c4c582f2faeee8fe52c9ad2ff241954966baacbcb9d2d5a0acd9f83"} Jul 07 13:15:52.500839 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.500529 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-866f955bdc-g8wjb" event={"ID":"3b69addc-9aea-42aa-a333-db21b736ba26","Type":"ContainerDied","Data":"3f7c0cfef5d34c79fcaf94506f0383909a8e58dacec7c32b51f04051e1daa75b"} Jul 07 13:15:52.500839 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.500549 2568 scope.go:117] "RemoveContainer" containerID="04725e0c3c4c582f2faeee8fe52c9ad2ff241954966baacbcb9d2d5a0acd9f83" Jul 07 13:15:52.509643 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.509621 2568 scope.go:117] "RemoveContainer" containerID="04725e0c3c4c582f2faeee8fe52c9ad2ff241954966baacbcb9d2d5a0acd9f83" Jul 07 13:15:52.509905 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:15:52.509884 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04725e0c3c4c582f2faeee8fe52c9ad2ff241954966baacbcb9d2d5a0acd9f83\": container with ID starting with 04725e0c3c4c582f2faeee8fe52c9ad2ff241954966baacbcb9d2d5a0acd9f83 not found: ID does not exist" containerID="04725e0c3c4c582f2faeee8fe52c9ad2ff241954966baacbcb9d2d5a0acd9f83" Jul 07 13:15:52.509966 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.509913 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04725e0c3c4c582f2faeee8fe52c9ad2ff241954966baacbcb9d2d5a0acd9f83"} err="failed to get container status \"04725e0c3c4c582f2faeee8fe52c9ad2ff241954966baacbcb9d2d5a0acd9f83\": rpc error: code = NotFound desc = could not find container \"04725e0c3c4c582f2faeee8fe52c9ad2ff241954966baacbcb9d2d5a0acd9f83\": container with ID starting with 04725e0c3c4c582f2faeee8fe52c9ad2ff241954966baacbcb9d2d5a0acd9f83 not found: ID does not exist" Jul 07 13:15:52.524360 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.524339 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-866f955bdc-g8wjb"] Jul 07 13:15:52.529617 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.529596 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-866f955bdc-g8wjb"] Jul 07 13:15:52.970538 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:52.970506 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b69addc-9aea-42aa-a333-db21b736ba26" path="/var/lib/kubelet/pods/3b69addc-9aea-42aa-a333-db21b736ba26/volumes" Jul 07 13:15:55.612424 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:55.612394 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:15:55.616152 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:15:55.616125 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-c8557b76-7gc7w" Jul 07 13:16:10.053243 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:10.053205 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:10.123001 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:10.122968 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:17.727339 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:17.727302 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:16:17.746248 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:17.746219 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:16:18.618372 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:18.618334 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 13:16:30.443653 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:30.443618 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:30.548837 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:30.548801 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:35.608200 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.608162 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-747c7c65d5-kbz84"] Jul 07 13:16:35.608640 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.608533 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b69addc-9aea-42aa-a333-db21b736ba26" containerName="console" Jul 07 13:16:35.608640 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.608546 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b69addc-9aea-42aa-a333-db21b736ba26" containerName="console" Jul 07 13:16:35.608640 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.608633 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b69addc-9aea-42aa-a333-db21b736ba26" containerName="console" Jul 07 13:16:35.611189 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.611167 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.615070 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.615047 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 13:16:35.615363 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.615331 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 13:16:35.615838 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.615669 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 13:16:35.616020 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.615761 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 13:16:35.616369 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.616348 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 13:16:35.616478 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.616354 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-4vhl6\"" Jul 07 13:16:35.628003 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.627949 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 13:16:35.631357 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.631333 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-747c7c65d5-kbz84"] Jul 07 13:16:35.738885 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.738854 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b5062415-d6c0-49c3-a582-28b98b6e5faf-telemeter-client-tls\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.738885 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.738886 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b5062415-d6c0-49c3-a582-28b98b6e5faf-secret-telemeter-client\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.739090 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.738907 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5062415-d6c0-49c3-a582-28b98b6e5faf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.739090 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.738933 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5062415-d6c0-49c3-a582-28b98b6e5faf-metrics-client-ca\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.739090 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.738969 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lt4n\" (UniqueName: \"kubernetes.io/projected/b5062415-d6c0-49c3-a582-28b98b6e5faf-kube-api-access-5lt4n\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.739090 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.739050 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5062415-d6c0-49c3-a582-28b98b6e5faf-serving-certs-ca-bundle\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.739090 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.739064 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b5062415-d6c0-49c3-a582-28b98b6e5faf-federate-client-tls\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.739090 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.739080 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5062415-d6c0-49c3-a582-28b98b6e5faf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.839876 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.839837 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b5062415-d6c0-49c3-a582-28b98b6e5faf-telemeter-client-tls\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.839876 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.839883 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b5062415-d6c0-49c3-a582-28b98b6e5faf-secret-telemeter-client\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.840117 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.839918 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5062415-d6c0-49c3-a582-28b98b6e5faf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.840117 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.839943 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5062415-d6c0-49c3-a582-28b98b6e5faf-metrics-client-ca\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.840117 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.839968 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lt4n\" (UniqueName: \"kubernetes.io/projected/b5062415-d6c0-49c3-a582-28b98b6e5faf-kube-api-access-5lt4n\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.840117 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.840064 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5062415-d6c0-49c3-a582-28b98b6e5faf-serving-certs-ca-bundle\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.840117 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.840086 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b5062415-d6c0-49c3-a582-28b98b6e5faf-federate-client-tls\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.840117 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.840112 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5062415-d6c0-49c3-a582-28b98b6e5faf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.840903 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.840868 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5062415-d6c0-49c3-a582-28b98b6e5faf-serving-certs-ca-bundle\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.841156 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.841132 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5062415-d6c0-49c3-a582-28b98b6e5faf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.841404 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.841377 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5062415-d6c0-49c3-a582-28b98b6e5faf-metrics-client-ca\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.842427 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.842396 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5062415-d6c0-49c3-a582-28b98b6e5faf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.842539 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.842524 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b5062415-d6c0-49c3-a582-28b98b6e5faf-telemeter-client-tls\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.842653 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.842636 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b5062415-d6c0-49c3-a582-28b98b6e5faf-secret-telemeter-client\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.843609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.842713 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b5062415-d6c0-49c3-a582-28b98b6e5faf-federate-client-tls\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.850296 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.850275 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lt4n\" (UniqueName: \"kubernetes.io/projected/b5062415-d6c0-49c3-a582-28b98b6e5faf-kube-api-access-5lt4n\") pod \"telemeter-client-747c7c65d5-kbz84\" (UID: \"b5062415-d6c0-49c3-a582-28b98b6e5faf\") " pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:35.923058 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:35.923026 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" Jul 07 13:16:36.071034 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:36.070995 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-747c7c65d5-kbz84"] Jul 07 13:16:36.075009 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:16:36.074982 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5062415_d6c0_49c3_a582_28b98b6e5faf.slice/crio-6b667d2209c378c1b3b23cff4ae34b29d2359f506d088640e3c2fb72438ac5f6 WatchSource:0}: Error finding container 6b667d2209c378c1b3b23cff4ae34b29d2359f506d088640e3c2fb72438ac5f6: Status 404 returned error can't find the container with id 6b667d2209c378c1b3b23cff4ae34b29d2359f506d088640e3c2fb72438ac5f6 Jul 07 13:16:36.670752 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:36.670717 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" event={"ID":"b5062415-d6c0-49c3-a582-28b98b6e5faf","Type":"ContainerStarted","Data":"6b667d2209c378c1b3b23cff4ae34b29d2359f506d088640e3c2fb72438ac5f6"} Jul 07 13:16:38.683163 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:38.683123 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" event={"ID":"b5062415-d6c0-49c3-a582-28b98b6e5faf","Type":"ContainerStarted","Data":"e449ad7b421548701bd3711c48887311b535e13d7f2acbb6b8fd533452021351"} Jul 07 13:16:38.683613 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:38.683188 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" event={"ID":"b5062415-d6c0-49c3-a582-28b98b6e5faf","Type":"ContainerStarted","Data":"fffba0ebb15d15998e3ccf3f67aa9baca227b5daf396479f864885a9a3c43b02"} Jul 07 13:16:38.683613 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:38.683209 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" event={"ID":"b5062415-d6c0-49c3-a582-28b98b6e5faf","Type":"ContainerStarted","Data":"b0ddae681350c97012fd8d65c6aabd3d5ad45601577c77ca96c355fc0f95b31c"} Jul 07 13:16:38.720218 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:38.720166 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-747c7c65d5-kbz84" podStartSLOduration=1.754249145 podStartE2EDuration="3.720149228s" totalImagesPullingTime="1.965900083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:16:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:38.71523131 +0000 UTC m=+156.393792640" watchObservedRunningTime="2026-07-07 13:16:38.720149228 +0000 UTC m=+156.398710560" Jul 07 13:16:39.358109 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:39.358066 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:40.004807 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:40.004775 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:40.086094 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:40.086063 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:45.730712 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:45.730678 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:45.798935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:45.798868 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:50.479041 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:50.479008 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bdb7f747d-r5kkp"] Jul 07 13:16:50.841692 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:50.841570 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:50.950753 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:50.950721 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:51.054035 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:51.053999 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:57.654070 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:57.653989 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:57.726090 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:16:57.726063 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:17:01.624855 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:01.624810 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:17:01.723049 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:01.723003 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:17:02.398955 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:02.398926 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:17:02.501506 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:02.501477 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:17:06.864879 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:06.864812 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:17:06.942288 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:06.942244 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:17:15.497969 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.497909 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-bdb7f747d-r5kkp" podUID="a60c7784-bf24-4c6b-8d17-d704afb646c0" containerName="console" containerID="cri-o://9bef97df1559feeef36b2bdbd1a3f7ba1529720c21b6a6edd1f5cf8692604f90" gracePeriod=15 Jul 07 13:17:15.739149 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.739127 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:17:15.773622 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.773541 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-oauth-config\") pod \"a60c7784-bf24-4c6b-8d17-d704afb646c0\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " Jul 07 13:17:15.773622 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.773600 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-trusted-ca-bundle\") pod \"a60c7784-bf24-4c6b-8d17-d704afb646c0\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " Jul 07 13:17:15.773779 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.773624 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-config\") pod \"a60c7784-bf24-4c6b-8d17-d704afb646c0\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " Jul 07 13:17:15.773779 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.773664 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-serving-cert\") pod \"a60c7784-bf24-4c6b-8d17-d704afb646c0\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " Jul 07 13:17:15.773779 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.773691 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v55xx\" (UniqueName: \"kubernetes.io/projected/a60c7784-bf24-4c6b-8d17-d704afb646c0-kube-api-access-v55xx\") pod \"a60c7784-bf24-4c6b-8d17-d704afb646c0\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " Jul 07 13:17:15.773779 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.773725 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-service-ca\") pod \"a60c7784-bf24-4c6b-8d17-d704afb646c0\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " Jul 07 13:17:15.773779 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.773767 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-oauth-serving-cert\") pod \"a60c7784-bf24-4c6b-8d17-d704afb646c0\" (UID: \"a60c7784-bf24-4c6b-8d17-d704afb646c0\") " Jul 07 13:17:15.774182 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.774151 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a60c7784-bf24-4c6b-8d17-d704afb646c0" (UID: "a60c7784-bf24-4c6b-8d17-d704afb646c0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:17:15.774294 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.774171 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-service-ca" (OuterVolumeSpecName: "service-ca") pod "a60c7784-bf24-4c6b-8d17-d704afb646c0" (UID: "a60c7784-bf24-4c6b-8d17-d704afb646c0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:17:15.774294 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.774079 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-config" (OuterVolumeSpecName: "console-config") pod "a60c7784-bf24-4c6b-8d17-d704afb646c0" (UID: "a60c7784-bf24-4c6b-8d17-d704afb646c0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:17:15.774294 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.774215 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a60c7784-bf24-4c6b-8d17-d704afb646c0" (UID: "a60c7784-bf24-4c6b-8d17-d704afb646c0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:17:15.775883 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.775860 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a60c7784-bf24-4c6b-8d17-d704afb646c0" (UID: "a60c7784-bf24-4c6b-8d17-d704afb646c0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:17:15.776034 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.776010 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a60c7784-bf24-4c6b-8d17-d704afb646c0" (UID: "a60c7784-bf24-4c6b-8d17-d704afb646c0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:17:15.776080 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.776056 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a60c7784-bf24-4c6b-8d17-d704afb646c0-kube-api-access-v55xx" (OuterVolumeSpecName: "kube-api-access-v55xx") pod "a60c7784-bf24-4c6b-8d17-d704afb646c0" (UID: "a60c7784-bf24-4c6b-8d17-d704afb646c0"). InnerVolumeSpecName "kube-api-access-v55xx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:17:15.833789 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.833756 2568 generic.go:356] "Generic (PLEG): container finished" podID="a60c7784-bf24-4c6b-8d17-d704afb646c0" containerID="9bef97df1559feeef36b2bdbd1a3f7ba1529720c21b6a6edd1f5cf8692604f90" exitCode=0 Jul 07 13:17:15.833940 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.833806 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bdb7f747d-r5kkp" Jul 07 13:17:15.833940 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.833838 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bdb7f747d-r5kkp" event={"ID":"a60c7784-bf24-4c6b-8d17-d704afb646c0","Type":"ContainerDied","Data":"9bef97df1559feeef36b2bdbd1a3f7ba1529720c21b6a6edd1f5cf8692604f90"} Jul 07 13:17:15.833940 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.833912 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bdb7f747d-r5kkp" event={"ID":"a60c7784-bf24-4c6b-8d17-d704afb646c0","Type":"ContainerDied","Data":"8126cd7801da395ce78046318ff6cadb3168edcf7a1b83018d2f10f18bb88a33"} Jul 07 13:17:15.833940 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.833938 2568 scope.go:117] "RemoveContainer" containerID="9bef97df1559feeef36b2bdbd1a3f7ba1529720c21b6a6edd1f5cf8692604f90" Jul 07 13:17:15.841825 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.841797 2568 scope.go:117] "RemoveContainer" containerID="9bef97df1559feeef36b2bdbd1a3f7ba1529720c21b6a6edd1f5cf8692604f90" Jul 07 13:17:15.842094 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:17:15.842046 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bef97df1559feeef36b2bdbd1a3f7ba1529720c21b6a6edd1f5cf8692604f90\": container with ID starting with 9bef97df1559feeef36b2bdbd1a3f7ba1529720c21b6a6edd1f5cf8692604f90 not found: ID does not exist" containerID="9bef97df1559feeef36b2bdbd1a3f7ba1529720c21b6a6edd1f5cf8692604f90" Jul 07 13:17:15.842094 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.842084 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bef97df1559feeef36b2bdbd1a3f7ba1529720c21b6a6edd1f5cf8692604f90"} err="failed to get container status \"9bef97df1559feeef36b2bdbd1a3f7ba1529720c21b6a6edd1f5cf8692604f90\": rpc error: code = NotFound desc = could not find container \"9bef97df1559feeef36b2bdbd1a3f7ba1529720c21b6a6edd1f5cf8692604f90\": container with ID starting with 9bef97df1559feeef36b2bdbd1a3f7ba1529720c21b6a6edd1f5cf8692604f90 not found: ID does not exist" Jul 07 13:17:15.871974 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.871946 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bdb7f747d-r5kkp"] Jul 07 13:17:15.874727 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.874707 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-oauth-config\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:17:15.874727 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.874728 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-trusted-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:17:15.874860 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.874738 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-config\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:17:15.874860 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.874746 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a60c7784-bf24-4c6b-8d17-d704afb646c0-console-serving-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:17:15.874860 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.874754 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v55xx\" (UniqueName: \"kubernetes.io/projected/a60c7784-bf24-4c6b-8d17-d704afb646c0-kube-api-access-v55xx\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:17:15.874860 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.874762 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-service-ca\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:17:15.874860 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.874770 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a60c7784-bf24-4c6b-8d17-d704afb646c0-oauth-serving-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:17:15.882303 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:15.882280 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bdb7f747d-r5kkp"] Jul 07 13:17:16.970157 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:16.970123 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a60c7784-bf24-4c6b-8d17-d704afb646c0" path="/var/lib/kubelet/pods/a60c7784-bf24-4c6b-8d17-d704afb646c0/volumes" Jul 07 13:17:39.296137 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:39.296102 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:17:45.207981 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:45.207948 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:17:45.275266 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:17:45.275236 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:04.432168 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:04.432126 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:04.591717 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:04.591649 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:23.000265 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:23.000235 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:25.975217 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:25.975140 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:26.865921 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:26.865887 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:29.831952 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:29.831918 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:29.942362 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:29.942328 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:31.918999 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:31.918961 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:32.042060 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:32.042032 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:32.376099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:32.376027 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:32.560860 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:32.560827 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:32.799559 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:32.799527 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r8jc4"] Jul 07 13:18:32.799891 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:32.799878 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a60c7784-bf24-4c6b-8d17-d704afb646c0" containerName="console" Jul 07 13:18:32.799929 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:32.799894 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a60c7784-bf24-4c6b-8d17-d704afb646c0" containerName="console" Jul 07 13:18:32.799961 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:32.799942 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a60c7784-bf24-4c6b-8d17-d704afb646c0" containerName="console" Jul 07 13:18:32.802257 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:32.802241 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-r8jc4" Jul 07 13:18:32.807563 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:32.807543 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 13:18:32.807943 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:32.807925 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-djplr\"" Jul 07 13:18:32.808478 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:32.808376 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 13:18:32.826467 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:32.826441 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r8jc4"] Jul 07 13:18:32.943445 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:32.943407 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6d4aef7-0161-415c-9f15-5f985775f0aa-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r8jc4\" (UID: \"e6d4aef7-0161-415c-9f15-5f985775f0aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r8jc4" Jul 07 13:18:32.943445 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:32.943449 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fcpn\" (UniqueName: \"kubernetes.io/projected/e6d4aef7-0161-415c-9f15-5f985775f0aa-kube-api-access-7fcpn\") pod \"cert-manager-webhook-6486ffd997-r8jc4\" (UID: \"e6d4aef7-0161-415c-9f15-5f985775f0aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r8jc4" Jul 07 13:18:33.044454 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:33.044361 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6d4aef7-0161-415c-9f15-5f985775f0aa-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r8jc4\" (UID: \"e6d4aef7-0161-415c-9f15-5f985775f0aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r8jc4" Jul 07 13:18:33.044646 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:33.044483 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fcpn\" (UniqueName: \"kubernetes.io/projected/e6d4aef7-0161-415c-9f15-5f985775f0aa-kube-api-access-7fcpn\") pod \"cert-manager-webhook-6486ffd997-r8jc4\" (UID: \"e6d4aef7-0161-415c-9f15-5f985775f0aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r8jc4" Jul 07 13:18:33.055116 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:33.055051 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fcpn\" (UniqueName: \"kubernetes.io/projected/e6d4aef7-0161-415c-9f15-5f985775f0aa-kube-api-access-7fcpn\") pod \"cert-manager-webhook-6486ffd997-r8jc4\" (UID: \"e6d4aef7-0161-415c-9f15-5f985775f0aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r8jc4" Jul 07 13:18:33.059028 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:33.059005 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6d4aef7-0161-415c-9f15-5f985775f0aa-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r8jc4\" (UID: \"e6d4aef7-0161-415c-9f15-5f985775f0aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r8jc4" Jul 07 13:18:33.121208 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:33.121160 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-r8jc4" Jul 07 13:18:33.253924 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:33.253901 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r8jc4"] Jul 07 13:18:33.256632 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:18:33.256606 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6d4aef7_0161_415c_9f15_5f985775f0aa.slice/crio-49ddf3d7cee94c73cc29c607b90d3df1f7117f0a62c814cd67ea39f24f08d5da WatchSource:0}: Error finding container 49ddf3d7cee94c73cc29c607b90d3df1f7117f0a62c814cd67ea39f24f08d5da: Status 404 returned error can't find the container with id 49ddf3d7cee94c73cc29c607b90d3df1f7117f0a62c814cd67ea39f24f08d5da Jul 07 13:18:34.138879 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:34.138843 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-r8jc4" event={"ID":"e6d4aef7-0161-415c-9f15-5f985775f0aa","Type":"ContainerStarted","Data":"49ddf3d7cee94c73cc29c607b90d3df1f7117f0a62c814cd67ea39f24f08d5da"} Jul 07 13:18:34.434910 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:34.434865 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:34.626021 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:34.625989 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:36.577968 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:36.577933 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:36.666201 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:36.666160 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:37.152594 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:37.152553 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-r8jc4" event={"ID":"e6d4aef7-0161-415c-9f15-5f985775f0aa","Type":"ContainerStarted","Data":"5cd1af4c60658cdb4daaca5825a254defbf681da5eea5f3e13656ffd17af1ca0"} Jul 07 13:18:37.152784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:37.152737 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-r8jc4" Jul 07 13:18:37.188711 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:37.188636 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-r8jc4" podStartSLOduration=2.320881517 podStartE2EDuration="5.188617446s" totalImagesPullingTime="2.867735929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:18:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:18:37.187017391 +0000 UTC m=+274.865578721" watchObservedRunningTime="2026-07-07 13:18:37.188617446 +0000 UTC m=+274.867178778" Jul 07 13:18:40.016802 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:40.016770 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-frrnp"] Jul 07 13:18:40.019519 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:40.019501 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-frrnp" Jul 07 13:18:40.025088 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:40.025068 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-dstfc\"" Jul 07 13:18:40.032756 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:40.032736 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-frrnp"] Jul 07 13:18:40.112003 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:40.111966 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2746b\" (UniqueName: \"kubernetes.io/projected/0f970600-abbb-4dc6-80e7-e89c91fb5077-kube-api-access-2746b\") pod \"cert-manager-cainjector-559cdd8c87-frrnp\" (UID: \"0f970600-abbb-4dc6-80e7-e89c91fb5077\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-frrnp" Jul 07 13:18:40.112168 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:40.112074 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f970600-abbb-4dc6-80e7-e89c91fb5077-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-frrnp\" (UID: \"0f970600-abbb-4dc6-80e7-e89c91fb5077\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-frrnp" Jul 07 13:18:40.212509 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:40.212471 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2746b\" (UniqueName: \"kubernetes.io/projected/0f970600-abbb-4dc6-80e7-e89c91fb5077-kube-api-access-2746b\") pod \"cert-manager-cainjector-559cdd8c87-frrnp\" (UID: \"0f970600-abbb-4dc6-80e7-e89c91fb5077\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-frrnp" Jul 07 13:18:40.212719 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:40.212562 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f970600-abbb-4dc6-80e7-e89c91fb5077-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-frrnp\" (UID: \"0f970600-abbb-4dc6-80e7-e89c91fb5077\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-frrnp" Jul 07 13:18:40.227441 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:40.227413 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f970600-abbb-4dc6-80e7-e89c91fb5077-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-frrnp\" (UID: \"0f970600-abbb-4dc6-80e7-e89c91fb5077\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-frrnp" Jul 07 13:18:40.236506 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:40.236484 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2746b\" (UniqueName: \"kubernetes.io/projected/0f970600-abbb-4dc6-80e7-e89c91fb5077-kube-api-access-2746b\") pod \"cert-manager-cainjector-559cdd8c87-frrnp\" (UID: \"0f970600-abbb-4dc6-80e7-e89c91fb5077\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-frrnp" Jul 07 13:18:40.328735 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:40.328661 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-frrnp" Jul 07 13:18:40.459962 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:40.459938 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-frrnp"] Jul 07 13:18:40.462191 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:18:40.462160 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f970600_abbb_4dc6_80e7_e89c91fb5077.slice/crio-7829da3ddc2119dff7abc75e9c04778b0a5975a5937e6b3b0e4ec4d9fbc2bcb2 WatchSource:0}: Error finding container 7829da3ddc2119dff7abc75e9c04778b0a5975a5937e6b3b0e4ec4d9fbc2bcb2: Status 404 returned error can't find the container with id 7829da3ddc2119dff7abc75e9c04778b0a5975a5937e6b3b0e4ec4d9fbc2bcb2 Jul 07 13:18:41.168164 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:41.168134 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-frrnp" event={"ID":"0f970600-abbb-4dc6-80e7-e89c91fb5077","Type":"ContainerStarted","Data":"b373a7d696c0f7615c8d84bdd5c71ec0c3d99491271828b75b2633622087d5b7"} Jul 07 13:18:41.168540 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:41.168176 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-frrnp" event={"ID":"0f970600-abbb-4dc6-80e7-e89c91fb5077","Type":"ContainerStarted","Data":"7829da3ddc2119dff7abc75e9c04778b0a5975a5937e6b3b0e4ec4d9fbc2bcb2"} Jul 07 13:18:41.190326 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:41.190277 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-frrnp" podStartSLOduration=2.190260846 podStartE2EDuration="2.190260846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:18:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:18:41.188946065 +0000 UTC m=+278.867507397" watchObservedRunningTime="2026-07-07 13:18:41.190260846 +0000 UTC m=+278.868822176" Jul 07 13:18:41.296225 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:41.296191 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:42.466703 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:42.466661 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:43.158107 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:43.158078 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-r8jc4" Jul 07 13:18:44.590690 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:44.590635 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:44.687312 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:44.687276 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:50.664881 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:50.664841 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:50.766915 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:50.766884 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:52.741154 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:18:52.741119 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:02.750049 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:02.750021 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 13:19:04.241242 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:04.241153 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:04.394999 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:04.394913 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:04.920099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:04.920071 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:05.173087 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.173004 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:05.302397 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.302360 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz"] Jul 07 13:19:05.304472 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.304442 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:05.305122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.305108 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.312050 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.312030 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 13:19:05.315481 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.315463 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 13:19:05.316307 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.316289 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 13:19:05.316394 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.316329 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 13:19:05.316394 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.316347 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ssqv6\"" Jul 07 13:19:05.330959 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.330939 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz"] Jul 07 13:19:05.434862 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.434758 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51740ce5-3d62-497a-b3ec-55ac5baf4708-tmp\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.434862 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.434821 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/51740ce5-3d62-497a-b3ec-55ac5baf4708-manifests\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.434862 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.434864 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/51740ce5-3d62-497a-b3ec-55ac5baf4708-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.435079 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.434939 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/51740ce5-3d62-497a-b3ec-55ac5baf4708-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.435079 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.434974 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkcz8\" (UniqueName: \"kubernetes.io/projected/51740ce5-3d62-497a-b3ec-55ac5baf4708-kube-api-access-qkcz8\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.443530 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.443502 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:05.535440 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.535404 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51740ce5-3d62-497a-b3ec-55ac5baf4708-tmp\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.535440 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.535445 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/51740ce5-3d62-497a-b3ec-55ac5baf4708-manifests\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.535730 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.535564 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/51740ce5-3d62-497a-b3ec-55ac5baf4708-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.535730 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.535660 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/51740ce5-3d62-497a-b3ec-55ac5baf4708-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.535730 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.535691 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkcz8\" (UniqueName: \"kubernetes.io/projected/51740ce5-3d62-497a-b3ec-55ac5baf4708-kube-api-access-qkcz8\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.535892 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.535804 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/51740ce5-3d62-497a-b3ec-55ac5baf4708-manifests\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.537990 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.537957 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/51740ce5-3d62-497a-b3ec-55ac5baf4708-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.538153 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.538135 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51740ce5-3d62-497a-b3ec-55ac5baf4708-tmp\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.538442 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.538420 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/51740ce5-3d62-497a-b3ec-55ac5baf4708-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.542406 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.542380 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:05.544820 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.544798 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkcz8\" (UniqueName: \"kubernetes.io/projected/51740ce5-3d62-497a-b3ec-55ac5baf4708-kube-api-access-qkcz8\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-8p8gz\" (UID: \"51740ce5-3d62-497a-b3ec-55ac5baf4708\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.614489 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.614464 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:05.646658 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.646629 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:05.743799 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.743766 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:05.754114 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.754025 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz"] Jul 07 13:19:05.756171 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:19:05.756141 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51740ce5_3d62_497a_b3ec_55ac5baf4708.slice/crio-5be5f268b647248080d394c7d1ce452e5297328e459512d7980fe8cb45a84b69 WatchSource:0}: Error finding container 5be5f268b647248080d394c7d1ce452e5297328e459512d7980fe8cb45a84b69: Status 404 returned error can't find the container with id 5be5f268b647248080d394c7d1ce452e5297328e459512d7980fe8cb45a84b69 Jul 07 13:19:05.758046 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.758029 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:19:05.979736 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:05.979660 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:06.265971 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:06.265895 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" event={"ID":"51740ce5-3d62-497a-b3ec-55ac5baf4708","Type":"ContainerStarted","Data":"5be5f268b647248080d394c7d1ce452e5297328e459512d7980fe8cb45a84b69"} Jul 07 13:19:09.281247 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:09.281208 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" event={"ID":"51740ce5-3d62-497a-b3ec-55ac5baf4708","Type":"ContainerStarted","Data":"590e86ff6bcae5375c4b9d23fbd94d3aa1e0d7111e0f660f43aa2572936dd040"} Jul 07 13:19:10.286761 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:10.286727 2568 generic.go:356] "Generic (PLEG): container finished" podID="51740ce5-3d62-497a-b3ec-55ac5baf4708" containerID="590e86ff6bcae5375c4b9d23fbd94d3aa1e0d7111e0f660f43aa2572936dd040" exitCode=0 Jul 07 13:19:10.287139 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:10.286817 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" event={"ID":"51740ce5-3d62-497a-b3ec-55ac5baf4708","Type":"ContainerDied","Data":"590e86ff6bcae5375c4b9d23fbd94d3aa1e0d7111e0f660f43aa2572936dd040"} Jul 07 13:19:11.294143 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:11.294110 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" event={"ID":"51740ce5-3d62-497a-b3ec-55ac5baf4708","Type":"ContainerStarted","Data":"0c1da544d6f87c5a3752f00970bbee37e42a3b3ec2a4c0a252c097f1e0142ce7"} Jul 07 13:19:11.294644 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:11.294241 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:12.364320 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:12.364263 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" podStartSLOduration=2.749546483 podStartE2EDuration="7.364244936s" totalImagesPullingTime="3.588050221s" totalInitContainerRuntime="1.026648232s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:12.362775321 +0000 UTC m=+310.041336653" watchObservedRunningTime="2026-07-07 13:19:12.364244936 +0000 UTC m=+310.042806265" Jul 07 13:19:12.376652 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:12.376622 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:12.888125 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:12.888083 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:22.300925 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:22.300893 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-8p8gz" Jul 07 13:19:24.619538 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:24.619503 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:24.776240 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:24.776203 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:25.065660 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:25.065053 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:25.211085 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:25.211053 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:25.468921 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:25.468888 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:25.580501 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:25.580470 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:25.958681 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:25.958548 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:26.653054 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:26.653019 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:26.872476 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:26.872432 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:26.981019 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:26.980984 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:28.183899 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:28.183859 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:31.279466 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:31.279433 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:31.375085 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:31.375039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:32.267562 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.267531 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z"] Jul 07 13:19:32.271092 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.271074 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.279010 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.278328 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-9vqhx\"" Jul 07 13:19:32.279010 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.278845 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 13:19:32.279010 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.278962 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 13:19:32.281385 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.281357 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 13:19:32.310486 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.310454 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z"] Jul 07 13:19:32.318920 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.318896 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:32.383493 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.383456 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a02dac82-c469-4f0a-a914-58459d0aec66-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.383652 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.383541 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.383652 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.383569 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a02dac82-c469-4f0a-a914-58459d0aec66-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.383652 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.383621 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.383776 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.383664 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t56mr\" (UniqueName: \"kubernetes.io/projected/a02dac82-c469-4f0a-a914-58459d0aec66-kube-api-access-t56mr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.383776 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.383688 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.383776 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.383706 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.383776 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.383728 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a02dac82-c469-4f0a-a914-58459d0aec66-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.383908 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.383776 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.484537 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.484474 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a02dac82-c469-4f0a-a914-58459d0aec66-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.484742 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.484600 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.484742 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.484629 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a02dac82-c469-4f0a-a914-58459d0aec66-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.484742 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.484656 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.484742 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.484690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t56mr\" (UniqueName: \"kubernetes.io/projected/a02dac82-c469-4f0a-a914-58459d0aec66-kube-api-access-t56mr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.484742 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.484711 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.484742 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.484736 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.485113 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.484756 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a02dac82-c469-4f0a-a914-58459d0aec66-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.485113 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.484803 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.485113 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.485042 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.485285 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.485147 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.485285 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.485240 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.485285 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.485267 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.485617 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.485597 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a02dac82-c469-4f0a-a914-58459d0aec66-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.486943 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.486921 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a02dac82-c469-4f0a-a914-58459d0aec66-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.487147 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.487126 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a02dac82-c469-4f0a-a914-58459d0aec66-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.512277 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.512245 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t56mr\" (UniqueName: \"kubernetes.io/projected/a02dac82-c469-4f0a-a914-58459d0aec66-kube-api-access-t56mr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.512953 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.512930 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a02dac82-c469-4f0a-a914-58459d0aec66-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4px87z\" (UID: \"a02dac82-c469-4f0a-a914-58459d0aec66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.520107 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.520032 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:32.582671 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.582639 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:32.653526 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.653503 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:32.723463 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:32.723440 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z"] Jul 07 13:19:32.725532 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:19:32.725507 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda02dac82_c469_4f0a_a914_58459d0aec66.slice/crio-3acb726693db0969d864f5a3a2604b669bb6a2ec8f8b81eabe8a3c3156de5c1a WatchSource:0}: Error finding container 3acb726693db0969d864f5a3a2604b669bb6a2ec8f8b81eabe8a3c3156de5c1a: Status 404 returned error can't find the container with id 3acb726693db0969d864f5a3a2604b669bb6a2ec8f8b81eabe8a3c3156de5c1a Jul 07 13:19:33.393758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.393722 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" event={"ID":"a02dac82-c469-4f0a-a914-58459d0aec66","Type":"ContainerStarted","Data":"3acb726693db0969d864f5a3a2604b669bb6a2ec8f8b81eabe8a3c3156de5c1a"} Jul 07 13:19:33.596322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.596200 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v"] Jul 07 13:19:33.599319 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.599290 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:33.612158 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.612133 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 13:19:33.612325 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.612133 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 13:19:33.612325 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.612133 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ps5lk\"" Jul 07 13:19:33.617553 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.617531 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 13:19:33.619033 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.619012 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 13:19:33.626708 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.626689 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 13:19:33.649993 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.649925 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v"] Jul 07 13:19:33.695937 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.695902 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/93fa4a10-6ee4-44f8-be51-5064bd32061d-manager-config\") pod \"lws-controller-manager-5b44df4f55-ndw7v\" (UID: \"93fa4a10-6ee4-44f8-be51-5064bd32061d\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:33.696123 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.695952 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/93fa4a10-6ee4-44f8-be51-5064bd32061d-cert\") pod \"lws-controller-manager-5b44df4f55-ndw7v\" (UID: \"93fa4a10-6ee4-44f8-be51-5064bd32061d\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:33.696123 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.695983 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/93fa4a10-6ee4-44f8-be51-5064bd32061d-metrics-cert\") pod \"lws-controller-manager-5b44df4f55-ndw7v\" (UID: \"93fa4a10-6ee4-44f8-be51-5064bd32061d\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:33.696123 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.696086 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv5gf\" (UniqueName: \"kubernetes.io/projected/93fa4a10-6ee4-44f8-be51-5064bd32061d-kube-api-access-lv5gf\") pod \"lws-controller-manager-5b44df4f55-ndw7v\" (UID: \"93fa4a10-6ee4-44f8-be51-5064bd32061d\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:33.797087 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.797042 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/93fa4a10-6ee4-44f8-be51-5064bd32061d-manager-config\") pod \"lws-controller-manager-5b44df4f55-ndw7v\" (UID: \"93fa4a10-6ee4-44f8-be51-5064bd32061d\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:33.797087 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.797086 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/93fa4a10-6ee4-44f8-be51-5064bd32061d-cert\") pod \"lws-controller-manager-5b44df4f55-ndw7v\" (UID: \"93fa4a10-6ee4-44f8-be51-5064bd32061d\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:33.797320 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.797118 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/93fa4a10-6ee4-44f8-be51-5064bd32061d-metrics-cert\") pod \"lws-controller-manager-5b44df4f55-ndw7v\" (UID: \"93fa4a10-6ee4-44f8-be51-5064bd32061d\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:33.797320 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.797182 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv5gf\" (UniqueName: \"kubernetes.io/projected/93fa4a10-6ee4-44f8-be51-5064bd32061d-kube-api-access-lv5gf\") pod \"lws-controller-manager-5b44df4f55-ndw7v\" (UID: \"93fa4a10-6ee4-44f8-be51-5064bd32061d\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:33.798415 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.798387 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/93fa4a10-6ee4-44f8-be51-5064bd32061d-manager-config\") pod \"lws-controller-manager-5b44df4f55-ndw7v\" (UID: \"93fa4a10-6ee4-44f8-be51-5064bd32061d\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:33.800534 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.800509 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/93fa4a10-6ee4-44f8-be51-5064bd32061d-metrics-cert\") pod \"lws-controller-manager-5b44df4f55-ndw7v\" (UID: \"93fa4a10-6ee4-44f8-be51-5064bd32061d\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:33.800655 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.800518 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/93fa4a10-6ee4-44f8-be51-5064bd32061d-cert\") pod \"lws-controller-manager-5b44df4f55-ndw7v\" (UID: \"93fa4a10-6ee4-44f8-be51-5064bd32061d\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:33.840360 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.840329 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv5gf\" (UniqueName: \"kubernetes.io/projected/93fa4a10-6ee4-44f8-be51-5064bd32061d-kube-api-access-lv5gf\") pod \"lws-controller-manager-5b44df4f55-ndw7v\" (UID: \"93fa4a10-6ee4-44f8-be51-5064bd32061d\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:33.913176 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:33.913095 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:34.093462 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:34.093436 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v"] Jul 07 13:19:34.096203 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:19:34.096159 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fa4a10_6ee4_44f8_be51_5064bd32061d.slice/crio-4494402fd4c78eeec03ed9ee2a815e09ada0ec5b123f9abdccc176cd68168de4 WatchSource:0}: Error finding container 4494402fd4c78eeec03ed9ee2a815e09ada0ec5b123f9abdccc176cd68168de4: Status 404 returned error can't find the container with id 4494402fd4c78eeec03ed9ee2a815e09ada0ec5b123f9abdccc176cd68168de4 Jul 07 13:19:34.400680 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:34.400641 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" event={"ID":"93fa4a10-6ee4-44f8-be51-5064bd32061d","Type":"ContainerStarted","Data":"4494402fd4c78eeec03ed9ee2a815e09ada0ec5b123f9abdccc176cd68168de4"} Jul 07 13:19:35.463368 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:35.463333 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:19:35.463611 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:35.463402 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:19:35.463611 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:35.463433 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:19:36.411847 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:36.411796 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" event={"ID":"a02dac82-c469-4f0a-a914-58459d0aec66","Type":"ContainerStarted","Data":"14a702b032a101f92dbd600a9bed17c53c1fb3a005b0a72bc30e29ace70fcb11"} Jul 07 13:19:36.471819 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:36.471762 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" podStartSLOduration=1.736379476 podStartE2EDuration="4.471745682s" totalImagesPullingTime="2.735366206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:36.466290348 +0000 UTC m=+334.144851683" watchObservedRunningTime="2026-07-07 13:19:36.471745682 +0000 UTC m=+334.150307012" Jul 07 13:19:36.583447 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:36.583404 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:36.589239 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:36.589209 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:37.041250 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.041210 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:37.219257 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.219229 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4"] Jul 07 13:19:37.219409 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.219364 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:37.221888 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.221869 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4" Jul 07 13:19:37.226105 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.226084 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 07 13:19:37.226216 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.226121 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-s4vb7\"" Jul 07 13:19:37.232520 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.232502 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs"] Jul 07 13:19:37.235838 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.235817 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs" Jul 07 13:19:37.241171 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.241150 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4"] Jul 07 13:19:37.242789 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.242766 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-62b9w\"" Jul 07 13:19:37.242891 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.242808 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 07 13:19:37.256783 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.256758 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs"] Jul 07 13:19:37.331010 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.330979 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c21c845-11bc-4b8d-bb88-7f94d3edcaa3-cert\") pod \"kserve-controller-manager-d6f4b65b8-xmrq4\" (UID: \"7c21c845-11bc-4b8d-bb88-7f94d3edcaa3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4" Jul 07 13:19:37.331146 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.331017 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2xpl\" (UniqueName: \"kubernetes.io/projected/7c21c845-11bc-4b8d-bb88-7f94d3edcaa3-kube-api-access-l2xpl\") pod \"kserve-controller-manager-d6f4b65b8-xmrq4\" (UID: \"7c21c845-11bc-4b8d-bb88-7f94d3edcaa3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4" Jul 07 13:19:37.331146 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.331110 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cafa8c81-4732-49fa-acb1-0687968d16e6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9s7zs\" (UID: \"cafa8c81-4732-49fa-acb1-0687968d16e6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs" Jul 07 13:19:37.331146 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.331136 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w69tf\" (UniqueName: \"kubernetes.io/projected/cafa8c81-4732-49fa-acb1-0687968d16e6-kube-api-access-w69tf\") pod \"llmisvc-controller-manager-777c6bd7d5-9s7zs\" (UID: \"cafa8c81-4732-49fa-acb1-0687968d16e6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs" Jul 07 13:19:37.378375 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.378350 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:37.419849 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.419810 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" event={"ID":"93fa4a10-6ee4-44f8-be51-5064bd32061d","Type":"ContainerStarted","Data":"376e1988e9482be700849178f88c11105710c257100b672424597562a3ba3bf0"} Jul 07 13:19:37.420063 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.420038 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:37.420201 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.420187 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:19:37.421266 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.421243 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4px87z" Jul 07 13:19:37.432089 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.432067 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c21c845-11bc-4b8d-bb88-7f94d3edcaa3-cert\") pod \"kserve-controller-manager-d6f4b65b8-xmrq4\" (UID: \"7c21c845-11bc-4b8d-bb88-7f94d3edcaa3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4" Jul 07 13:19:37.432178 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.432098 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2xpl\" (UniqueName: \"kubernetes.io/projected/7c21c845-11bc-4b8d-bb88-7f94d3edcaa3-kube-api-access-l2xpl\") pod \"kserve-controller-manager-d6f4b65b8-xmrq4\" (UID: \"7c21c845-11bc-4b8d-bb88-7f94d3edcaa3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4" Jul 07 13:19:37.432178 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.432137 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cafa8c81-4732-49fa-acb1-0687968d16e6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9s7zs\" (UID: \"cafa8c81-4732-49fa-acb1-0687968d16e6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs" Jul 07 13:19:37.432178 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.432156 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w69tf\" (UniqueName: \"kubernetes.io/projected/cafa8c81-4732-49fa-acb1-0687968d16e6-kube-api-access-w69tf\") pod \"llmisvc-controller-manager-777c6bd7d5-9s7zs\" (UID: \"cafa8c81-4732-49fa-acb1-0687968d16e6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs" Jul 07 13:19:37.432294 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:19:37.432209 2568 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 07 13:19:37.432294 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:19:37.432238 2568 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 07 13:19:37.432294 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:19:37.432270 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c21c845-11bc-4b8d-bb88-7f94d3edcaa3-cert podName:7c21c845-11bc-4b8d-bb88-7f94d3edcaa3 nodeName:}" failed. No retries permitted until 2026-07-07 13:19:37.932254646 +0000 UTC m=+335.610815956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7c21c845-11bc-4b8d-bb88-7f94d3edcaa3-cert") pod "kserve-controller-manager-d6f4b65b8-xmrq4" (UID: "7c21c845-11bc-4b8d-bb88-7f94d3edcaa3") : secret "kserve-webhook-server-cert" not found Jul 07 13:19:37.432405 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:19:37.432294 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cafa8c81-4732-49fa-acb1-0687968d16e6-cert podName:cafa8c81-4732-49fa-acb1-0687968d16e6 nodeName:}" failed. No retries permitted until 2026-07-07 13:19:37.93227869 +0000 UTC m=+335.610840012 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cafa8c81-4732-49fa-acb1-0687968d16e6-cert") pod "llmisvc-controller-manager-777c6bd7d5-9s7zs" (UID: "cafa8c81-4732-49fa-acb1-0687968d16e6") : secret "llmisvc-webhook-server-cert" not found Jul 07 13:19:37.465976 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.465948 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w69tf\" (UniqueName: \"kubernetes.io/projected/cafa8c81-4732-49fa-acb1-0687968d16e6-kube-api-access-w69tf\") pod \"llmisvc-controller-manager-777c6bd7d5-9s7zs\" (UID: \"cafa8c81-4732-49fa-acb1-0687968d16e6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs" Jul 07 13:19:37.479468 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.479445 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2xpl\" (UniqueName: \"kubernetes.io/projected/7c21c845-11bc-4b8d-bb88-7f94d3edcaa3-kube-api-access-l2xpl\") pod \"kserve-controller-manager-d6f4b65b8-xmrq4\" (UID: \"7c21c845-11bc-4b8d-bb88-7f94d3edcaa3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4" Jul 07 13:19:37.621333 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.621217 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" podStartSLOduration=1.505703448 podStartE2EDuration="4.62118929s" totalImagesPullingTime="3.115485842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:37.524507125 +0000 UTC m=+335.203068467" watchObservedRunningTime="2026-07-07 13:19:37.62118929 +0000 UTC m=+335.299750622" Jul 07 13:19:37.728253 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.728088 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:37.850874 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.850804 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:37.937944 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.937911 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c21c845-11bc-4b8d-bb88-7f94d3edcaa3-cert\") pod \"kserve-controller-manager-d6f4b65b8-xmrq4\" (UID: \"7c21c845-11bc-4b8d-bb88-7f94d3edcaa3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4" Jul 07 13:19:37.938101 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.937975 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cafa8c81-4732-49fa-acb1-0687968d16e6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9s7zs\" (UID: \"cafa8c81-4732-49fa-acb1-0687968d16e6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs" Jul 07 13:19:37.940281 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.940259 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cafa8c81-4732-49fa-acb1-0687968d16e6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9s7zs\" (UID: \"cafa8c81-4732-49fa-acb1-0687968d16e6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs" Jul 07 13:19:37.940281 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:37.940269 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c21c845-11bc-4b8d-bb88-7f94d3edcaa3-cert\") pod \"kserve-controller-manager-d6f4b65b8-xmrq4\" (UID: \"7c21c845-11bc-4b8d-bb88-7f94d3edcaa3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4" Jul 07 13:19:38.219229 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:38.219159 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4" Jul 07 13:19:38.239272 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:38.239249 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs" Jul 07 13:19:38.382591 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:38.382551 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4"] Jul 07 13:19:38.384585 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:19:38.384536 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c21c845_11bc_4b8d_bb88_7f94d3edcaa3.slice/crio-6b85a01d81443337e08502db753a457bfa5e8081cedb913ddacad9020cd52bad WatchSource:0}: Error finding container 6b85a01d81443337e08502db753a457bfa5e8081cedb913ddacad9020cd52bad: Status 404 returned error can't find the container with id 6b85a01d81443337e08502db753a457bfa5e8081cedb913ddacad9020cd52bad Jul 07 13:19:38.402867 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:38.402845 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs"] Jul 07 13:19:38.404355 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:19:38.404326 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcafa8c81_4732_49fa_acb1_0687968d16e6.slice/crio-30dd21f03d4d0528b1beec6310b6833a6b99cbdd2b388118a6f7daee6a1758f5 WatchSource:0}: Error finding container 30dd21f03d4d0528b1beec6310b6833a6b99cbdd2b388118a6f7daee6a1758f5: Status 404 returned error can't find the container with id 30dd21f03d4d0528b1beec6310b6833a6b99cbdd2b388118a6f7daee6a1758f5 Jul 07 13:19:38.424934 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:38.424910 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4" event={"ID":"7c21c845-11bc-4b8d-bb88-7f94d3edcaa3","Type":"ContainerStarted","Data":"6b85a01d81443337e08502db753a457bfa5e8081cedb913ddacad9020cd52bad"} Jul 07 13:19:38.425946 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:38.425926 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs" event={"ID":"cafa8c81-4732-49fa-acb1-0687968d16e6","Type":"ContainerStarted","Data":"30dd21f03d4d0528b1beec6310b6833a6b99cbdd2b388118a6f7daee6a1758f5"} Jul 07 13:19:41.442203 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:41.442169 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4" event={"ID":"7c21c845-11bc-4b8d-bb88-7f94d3edcaa3","Type":"ContainerStarted","Data":"78f1c3258e9a3e48281ae2918dfecb2397738e755f8f75db249d76218a481a42"} Jul 07 13:19:41.442554 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:41.442306 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4" Jul 07 13:19:41.470380 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:41.470322 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4" podStartSLOduration=1.504704159 podStartE2EDuration="4.470305241s" totalImagesPullingTime="2.965601082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:41.468056014 +0000 UTC m=+339.146617345" watchObservedRunningTime="2026-07-07 13:19:41.470305241 +0000 UTC m=+339.148866577" Jul 07 13:19:42.448266 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:42.448172 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs" event={"ID":"cafa8c81-4732-49fa-acb1-0687968d16e6","Type":"ContainerStarted","Data":"f9c472ed12014886f648e2a308ba64f2d11596eabf691c99ec4dadd952033027"} Jul 07 13:19:42.448626 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:42.448463 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs" Jul 07 13:19:42.485286 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:42.485231 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs" podStartSLOduration=1.717357962 podStartE2EDuration="5.485217314s" totalImagesPullingTime="3.767859352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:42.48408486 +0000 UTC m=+340.162646192" watchObservedRunningTime="2026-07-07 13:19:42.485217314 +0000 UTC m=+340.163778645" Jul 07 13:19:48.428751 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:19:48.428716 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-ndw7v" Jul 07 13:20:12.452881 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:12.452852 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xmrq4" Jul 07 13:20:13.455201 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:13.455169 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9s7zs" Jul 07 13:20:26.289207 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:26.289170 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:20:26.389361 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:26.389322 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:20:26.547128 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:26.547037 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:20:27.227311 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:27.227266 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:20:27.432529 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:27.432501 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:20:34.546666 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:34.546628 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:20:35.547667 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:35.547632 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:20:35.638037 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:35.637989 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:20:51.372611 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:51.372548 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:20:52.009289 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:52.009261 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:20:57.119270 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:57.119240 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:20:59.753027 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:59.752992 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rh22p"] Jul 07 13:20:59.755585 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:59.755555 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rh22p" Jul 07 13:20:59.760352 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:59.760329 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 13:20:59.760491 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:59.760355 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 07 13:20:59.760733 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:59.760717 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 13:20:59.761670 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:59.761649 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-2flnv\"" Jul 07 13:20:59.773526 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:59.773507 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rh22p"] Jul 07 13:20:59.838767 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:59.838742 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lbrk\" (UniqueName: \"kubernetes.io/projected/f438c79b-b855-4fea-94bb-e3c1affe9d6d-kube-api-access-5lbrk\") pod \"dns-operator-controller-manager-864dd78c4c-rh22p\" (UID: \"f438c79b-b855-4fea-94bb-e3c1affe9d6d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rh22p" Jul 07 13:20:59.940089 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:59.940057 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lbrk\" (UniqueName: \"kubernetes.io/projected/f438c79b-b855-4fea-94bb-e3c1affe9d6d-kube-api-access-5lbrk\") pod \"dns-operator-controller-manager-864dd78c4c-rh22p\" (UID: \"f438c79b-b855-4fea-94bb-e3c1affe9d6d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rh22p" Jul 07 13:20:59.950313 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:20:59.950290 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lbrk\" (UniqueName: \"kubernetes.io/projected/f438c79b-b855-4fea-94bb-e3c1affe9d6d-kube-api-access-5lbrk\") pod \"dns-operator-controller-manager-864dd78c4c-rh22p\" (UID: \"f438c79b-b855-4fea-94bb-e3c1affe9d6d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rh22p" Jul 07 13:21:00.065492 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:00.065412 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rh22p" Jul 07 13:21:00.220386 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:00.220326 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rh22p"] Jul 07 13:21:00.224585 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:21:00.224545 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf438c79b_b855_4fea_94bb_e3c1affe9d6d.slice/crio-79cc4d995bf7b42dcf3fa3f67b9cc67b1bf9e2721b7d56395ddae6c8d7114c8b WatchSource:0}: Error finding container 79cc4d995bf7b42dcf3fa3f67b9cc67b1bf9e2721b7d56395ddae6c8d7114c8b: Status 404 returned error can't find the container with id 79cc4d995bf7b42dcf3fa3f67b9cc67b1bf9e2721b7d56395ddae6c8d7114c8b Jul 07 13:21:00.810784 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:00.810747 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rh22p" event={"ID":"f438c79b-b855-4fea-94bb-e3c1affe9d6d","Type":"ContainerStarted","Data":"79cc4d995bf7b42dcf3fa3f67b9cc67b1bf9e2721b7d56395ddae6c8d7114c8b"} Jul 07 13:21:01.208568 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:01.208537 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:02.821707 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:02.821620 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rh22p" event={"ID":"f438c79b-b855-4fea-94bb-e3c1affe9d6d","Type":"ContainerStarted","Data":"1c3cb9913ad6ea50977cf926066a83d4be43ca48f3b6e17b98f30e7528e25610"} Jul 07 13:21:02.822070 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:02.821760 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rh22p" Jul 07 13:21:03.844714 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:03.844664 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rh22p" podStartSLOduration=2.5771684969999997 podStartE2EDuration="4.844649054s" totalImagesPullingTime="2.267480557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:20:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:21:02.855843328 +0000 UTC m=+420.534404659" watchObservedRunningTime="2026-07-07 13:21:03.844649054 +0000 UTC m=+421.523210390" Jul 07 13:21:03.845722 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:03.845691 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s"] Jul 07 13:21:03.848258 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:03.848243 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:03.856618 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:03.856600 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-s9znd\"" Jul 07 13:21:03.935548 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:03.935514 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s"] Jul 07 13:21:03.975208 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:03.975181 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqn59\" (UniqueName: \"kubernetes.io/projected/0d0b0181-ce71-4d74-be51-45992d8264b2-kube-api-access-bqn59\") pod \"limitador-operator-controller-manager-859d7b78bd-cg46s\" (UID: \"0d0b0181-ce71-4d74-be51-45992d8264b2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:04.076539 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:04.076490 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqn59\" (UniqueName: \"kubernetes.io/projected/0d0b0181-ce71-4d74-be51-45992d8264b2-kube-api-access-bqn59\") pod \"limitador-operator-controller-manager-859d7b78bd-cg46s\" (UID: \"0d0b0181-ce71-4d74-be51-45992d8264b2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:04.093745 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:04.093714 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqn59\" (UniqueName: \"kubernetes.io/projected/0d0b0181-ce71-4d74-be51-45992d8264b2-kube-api-access-bqn59\") pod \"limitador-operator-controller-manager-859d7b78bd-cg46s\" (UID: \"0d0b0181-ce71-4d74-be51-45992d8264b2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:04.158869 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:04.158831 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:04.294788 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:04.294767 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s"] Jul 07 13:21:04.297010 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:21:04.296986 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d0b0181_ce71_4d74_be51_45992d8264b2.slice/crio-68541e3591713fd3ba8d37aca3ee8df0dc9de63a26f3e09a32825884d44f9237 WatchSource:0}: Error finding container 68541e3591713fd3ba8d37aca3ee8df0dc9de63a26f3e09a32825884d44f9237: Status 404 returned error can't find the container with id 68541e3591713fd3ba8d37aca3ee8df0dc9de63a26f3e09a32825884d44f9237 Jul 07 13:21:04.604956 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:04.604908 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:04.831917 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:04.831883 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" event={"ID":"0d0b0181-ce71-4d74-be51-45992d8264b2","Type":"ContainerStarted","Data":"68541e3591713fd3ba8d37aca3ee8df0dc9de63a26f3e09a32825884d44f9237"} Jul 07 13:21:06.843672 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:06.843634 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" event={"ID":"0d0b0181-ce71-4d74-be51-45992d8264b2","Type":"ContainerStarted","Data":"cb0e3738b4acfad27983f880cde4da33491cedd0d33c0751f3e07b1aa49516ba"} Jul 07 13:21:06.844060 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:06.843727 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:06.865235 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:06.865185 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" podStartSLOduration=1.4336284209999999 podStartE2EDuration="3.865162585s" totalImagesPullingTime="2.431534164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:21:06.8649646 +0000 UTC m=+424.543525931" watchObservedRunningTime="2026-07-07 13:21:06.865162585 +0000 UTC m=+424.543723915" Jul 07 13:21:13.828192 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:13.828158 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rh22p" Jul 07 13:21:17.851262 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:17.851234 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:20.625774 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.625736 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b"] Jul 07 13:21:20.628790 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.628766 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" Jul 07 13:21:20.638217 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.638197 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 07 13:21:20.643276 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.643257 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 07 13:21:20.643374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.643285 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7jgsl\"" Jul 07 13:21:20.670164 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.670145 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b"] Jul 07 13:21:20.679278 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.679255 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:20.718275 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.718252 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/311cc01e-312f-4240-8ac4-57edace3ea5f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-b6t7b\" (UID: \"311cc01e-312f-4240-8ac4-57edace3ea5f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" Jul 07 13:21:20.718390 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.718322 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/311cc01e-312f-4240-8ac4-57edace3ea5f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-b6t7b\" (UID: \"311cc01e-312f-4240-8ac4-57edace3ea5f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" Jul 07 13:21:20.718390 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.718365 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrbpw\" (UniqueName: \"kubernetes.io/projected/311cc01e-312f-4240-8ac4-57edace3ea5f-kube-api-access-hrbpw\") pod \"kuadrant-console-plugin-66b5774985-b6t7b\" (UID: \"311cc01e-312f-4240-8ac4-57edace3ea5f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" Jul 07 13:21:20.819026 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.818994 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/311cc01e-312f-4240-8ac4-57edace3ea5f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-b6t7b\" (UID: \"311cc01e-312f-4240-8ac4-57edace3ea5f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" Jul 07 13:21:20.819183 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.819052 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/311cc01e-312f-4240-8ac4-57edace3ea5f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-b6t7b\" (UID: \"311cc01e-312f-4240-8ac4-57edace3ea5f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" Jul 07 13:21:20.819183 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.819087 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrbpw\" (UniqueName: \"kubernetes.io/projected/311cc01e-312f-4240-8ac4-57edace3ea5f-kube-api-access-hrbpw\") pod \"kuadrant-console-plugin-66b5774985-b6t7b\" (UID: \"311cc01e-312f-4240-8ac4-57edace3ea5f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" Jul 07 13:21:20.819183 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:21:20.819133 2568 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 07 13:21:20.819283 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:21:20.819213 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311cc01e-312f-4240-8ac4-57edace3ea5f-plugin-serving-cert podName:311cc01e-312f-4240-8ac4-57edace3ea5f nodeName:}" failed. No retries permitted until 2026-07-07 13:21:21.319194375 +0000 UTC m=+438.997755698 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/311cc01e-312f-4240-8ac4-57edace3ea5f-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-b6t7b" (UID: "311cc01e-312f-4240-8ac4-57edace3ea5f") : secret "plugin-serving-cert" not found Jul 07 13:21:20.819780 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.819764 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/311cc01e-312f-4240-8ac4-57edace3ea5f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-b6t7b\" (UID: \"311cc01e-312f-4240-8ac4-57edace3ea5f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" Jul 07 13:21:20.837708 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.837681 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrbpw\" (UniqueName: \"kubernetes.io/projected/311cc01e-312f-4240-8ac4-57edace3ea5f-kube-api-access-hrbpw\") pod \"kuadrant-console-plugin-66b5774985-b6t7b\" (UID: \"311cc01e-312f-4240-8ac4-57edace3ea5f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" Jul 07 13:21:20.985150 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:20.985125 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:21.322015 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:21.321945 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/311cc01e-312f-4240-8ac4-57edace3ea5f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-b6t7b\" (UID: \"311cc01e-312f-4240-8ac4-57edace3ea5f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" Jul 07 13:21:21.324305 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:21.324285 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/311cc01e-312f-4240-8ac4-57edace3ea5f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-b6t7b\" (UID: \"311cc01e-312f-4240-8ac4-57edace3ea5f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" Jul 07 13:21:21.537682 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:21.537655 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" Jul 07 13:21:21.678822 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:21.678799 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b"] Jul 07 13:21:21.679927 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:21:21.679899 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod311cc01e_312f_4240_8ac4_57edace3ea5f.slice/crio-6314f39bb3ea0c8cbc85d831ff5d297345c4324e78b5248042697daf909f1f9b WatchSource:0}: Error finding container 6314f39bb3ea0c8cbc85d831ff5d297345c4324e78b5248042697daf909f1f9b: Status 404 returned error can't find the container with id 6314f39bb3ea0c8cbc85d831ff5d297345c4324e78b5248042697daf909f1f9b Jul 07 13:21:21.914891 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:21.914862 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" event={"ID":"311cc01e-312f-4240-8ac4-57edace3ea5f","Type":"ContainerStarted","Data":"6314f39bb3ea0c8cbc85d831ff5d297345c4324e78b5248042697daf909f1f9b"} Jul 07 13:21:23.201263 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:23.201220 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:23.442198 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:23.442154 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:24.733418 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:24.733301 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:25.699433 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:25.699396 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:26.848587 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:26.848534 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:30.610717 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:30.610685 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt"] Jul 07 13:21:30.620291 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:30.620263 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" Jul 07 13:21:30.623925 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:30.623892 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mszc8\"" Jul 07 13:21:30.646030 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:30.645994 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s"] Jul 07 13:21:30.646452 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:30.646260 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" podUID="0d0b0181-ce71-4d74-be51-45992d8264b2" containerName="manager" containerID="cri-o://cb0e3738b4acfad27983f880cde4da33491cedd0d33c0751f3e07b1aa49516ba" gracePeriod=2 Jul 07 13:21:30.649671 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:30.649609 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt"] Jul 07 13:21:30.652297 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:30.652273 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s"] Jul 07 13:21:30.672490 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:21:30.672459 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cg46s\" is forbidden: User \"system:node:ip-10-0-136-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-215.ec2.internal' and this object" podUID="0d0b0181-ce71-4d74-be51-45992d8264b2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:30.718537 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:30.718482 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/674be9eb-18ed-47f6-a0be-948f0fa0f80c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jcxgt\" (UID: \"674be9eb-18ed-47f6-a0be-948f0fa0f80c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" Jul 07 13:21:30.718712 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:30.718682 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2qhj\" (UniqueName: \"kubernetes.io/projected/674be9eb-18ed-47f6-a0be-948f0fa0f80c-kube-api-access-w2qhj\") pod \"kuadrant-operator-controller-manager-55467d9f7-jcxgt\" (UID: \"674be9eb-18ed-47f6-a0be-948f0fa0f80c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" Jul 07 13:21:30.819852 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:30.819814 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/674be9eb-18ed-47f6-a0be-948f0fa0f80c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jcxgt\" (UID: \"674be9eb-18ed-47f6-a0be-948f0fa0f80c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" Jul 07 13:21:30.820020 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:30.819927 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2qhj\" (UniqueName: \"kubernetes.io/projected/674be9eb-18ed-47f6-a0be-948f0fa0f80c-kube-api-access-w2qhj\") pod \"kuadrant-operator-controller-manager-55467d9f7-jcxgt\" (UID: \"674be9eb-18ed-47f6-a0be-948f0fa0f80c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" Jul 07 13:21:30.820247 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:30.820221 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/674be9eb-18ed-47f6-a0be-948f0fa0f80c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jcxgt\" (UID: \"674be9eb-18ed-47f6-a0be-948f0fa0f80c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" Jul 07 13:21:30.836376 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:30.836346 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2qhj\" (UniqueName: \"kubernetes.io/projected/674be9eb-18ed-47f6-a0be-948f0fa0f80c-kube-api-access-w2qhj\") pod \"kuadrant-operator-controller-manager-55467d9f7-jcxgt\" (UID: \"674be9eb-18ed-47f6-a0be-948f0fa0f80c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" Jul 07 13:21:30.935611 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:30.935553 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" Jul 07 13:21:32.005984 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:32.005946 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:32.284140 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:32.284058 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:33.031704 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:33.031664 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:33.126199 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:33.126154 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:33.400358 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:33.400267 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:33.500380 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:33.500350 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:37.073324 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:37.073291 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:37.188286 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:37.188257 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:41.059990 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:41.059955 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:41.167527 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:41.167477 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:43.156200 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:43.156124 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:43.257954 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:43.257917 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:44.257392 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.257371 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:44.264433 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:21:44.264400 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cg46s\" is forbidden: User \"system:node:ip-10-0-136-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-215.ec2.internal' and this object" podUID="0d0b0181-ce71-4d74-be51-45992d8264b2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:44.264433 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.264412 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt"] Jul 07 13:21:44.264682 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:21:44.264655 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod674be9eb_18ed_47f6_a0be_948f0fa0f80c.slice/crio-583f029eab86118940731ccd910645b9ddfb413af5f9e685e445ed21ede571ed WatchSource:0}: Error finding container 583f029eab86118940731ccd910645b9ddfb413af5f9e685e445ed21ede571ed: Status 404 returned error can't find the container with id 583f029eab86118940731ccd910645b9ddfb413af5f9e685e445ed21ede571ed Jul 07 13:21:44.356737 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.356704 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqn59\" (UniqueName: \"kubernetes.io/projected/0d0b0181-ce71-4d74-be51-45992d8264b2-kube-api-access-bqn59\") pod \"0d0b0181-ce71-4d74-be51-45992d8264b2\" (UID: \"0d0b0181-ce71-4d74-be51-45992d8264b2\") " Jul 07 13:21:44.358813 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.358788 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d0b0181-ce71-4d74-be51-45992d8264b2-kube-api-access-bqn59" (OuterVolumeSpecName: "kube-api-access-bqn59") pod "0d0b0181-ce71-4d74-be51-45992d8264b2" (UID: "0d0b0181-ce71-4d74-be51-45992d8264b2"). InnerVolumeSpecName "kube-api-access-bqn59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:21:44.457441 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.457399 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqn59\" (UniqueName: \"kubernetes.io/projected/0d0b0181-ce71-4d74-be51-45992d8264b2-kube-api-access-bqn59\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:21:44.641451 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.641422 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5797dc48b7-8jsrb"] Jul 07 13:21:44.641882 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.641866 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d0b0181-ce71-4d74-be51-45992d8264b2" containerName="manager" Jul 07 13:21:44.641933 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.641884 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d0b0181-ce71-4d74-be51-45992d8264b2" containerName="manager" Jul 07 13:21:44.641966 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.641955 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d0b0181-ce71-4d74-be51-45992d8264b2" containerName="manager" Jul 07 13:21:44.644509 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.644493 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.653983 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:21:44.653949 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cg46s\" is forbidden: User \"system:node:ip-10-0-136-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-215.ec2.internal' and this object" podUID="0d0b0181-ce71-4d74-be51-45992d8264b2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:44.653983 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.653975 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 13:21:44.654120 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.653975 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 13:21:44.654120 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.653982 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 13:21:44.655445 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.655427 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 13:21:44.655445 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.655427 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 13:21:44.655650 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.655489 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 13:21:44.655650 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.655491 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 13:21:44.655650 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.655558 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dkwl9\"" Jul 07 13:21:44.666695 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.666676 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5797dc48b7-8jsrb"] Jul 07 13:21:44.676910 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.676888 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 13:21:44.719190 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.719110 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:44.760425 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.760386 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4345a50e-990d-4242-a0f1-948b1257f288-trusted-ca-bundle\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.760619 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.760455 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4345a50e-990d-4242-a0f1-948b1257f288-console-serving-cert\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.760619 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.760484 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4345a50e-990d-4242-a0f1-948b1257f288-console-config\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.760619 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.760508 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4345a50e-990d-4242-a0f1-948b1257f288-console-oauth-config\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.760619 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.760529 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4345a50e-990d-4242-a0f1-948b1257f288-oauth-serving-cert\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.760619 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.760558 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g77kt\" (UniqueName: \"kubernetes.io/projected/4345a50e-990d-4242-a0f1-948b1257f288-kube-api-access-g77kt\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.760919 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.760671 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4345a50e-990d-4242-a0f1-948b1257f288-service-ca\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.820402 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.820367 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:44.862204 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.861549 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4345a50e-990d-4242-a0f1-948b1257f288-service-ca\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.862204 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.861644 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4345a50e-990d-4242-a0f1-948b1257f288-trusted-ca-bundle\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.862204 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.861718 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4345a50e-990d-4242-a0f1-948b1257f288-console-serving-cert\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.862204 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.861749 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4345a50e-990d-4242-a0f1-948b1257f288-console-config\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.862204 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.861781 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4345a50e-990d-4242-a0f1-948b1257f288-console-oauth-config\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.862204 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.861806 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4345a50e-990d-4242-a0f1-948b1257f288-oauth-serving-cert\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.862204 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.861835 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g77kt\" (UniqueName: \"kubernetes.io/projected/4345a50e-990d-4242-a0f1-948b1257f288-kube-api-access-g77kt\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.862679 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.862356 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4345a50e-990d-4242-a0f1-948b1257f288-service-ca\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.862679 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.862429 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4345a50e-990d-4242-a0f1-948b1257f288-console-config\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.862679 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.862647 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4345a50e-990d-4242-a0f1-948b1257f288-trusted-ca-bundle\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.862823 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.862765 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4345a50e-990d-4242-a0f1-948b1257f288-oauth-serving-cert\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.864665 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.864499 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4345a50e-990d-4242-a0f1-948b1257f288-console-serving-cert\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.864787 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.864729 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4345a50e-990d-4242-a0f1-948b1257f288-console-oauth-config\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.872364 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.872337 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g77kt\" (UniqueName: \"kubernetes.io/projected/4345a50e-990d-4242-a0f1-948b1257f288-kube-api-access-g77kt\") pod \"console-5797dc48b7-8jsrb\" (UID: \"4345a50e-990d-4242-a0f1-948b1257f288\") " pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.953850 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.953814 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:44.970669 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:44.970610 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d0b0181-ce71-4d74-be51-45992d8264b2" path="/var/lib/kubelet/pods/0d0b0181-ce71-4d74-be51-45992d8264b2/volumes" Jul 07 13:21:45.041216 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:45.041174 2568 generic.go:356] "Generic (PLEG): container finished" podID="0d0b0181-ce71-4d74-be51-45992d8264b2" containerID="cb0e3738b4acfad27983f880cde4da33491cedd0d33c0751f3e07b1aa49516ba" exitCode=0 Jul 07 13:21:45.041376 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:45.041269 2568 scope.go:117] "RemoveContainer" containerID="cb0e3738b4acfad27983f880cde4da33491cedd0d33c0751f3e07b1aa49516ba" Jul 07 13:21:45.041433 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:45.041426 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:45.044730 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:21:45.044702 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cg46s\" is forbidden: User \"system:node:ip-10-0-136-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-215.ec2.internal' and this object" podUID="0d0b0181-ce71-4d74-be51-45992d8264b2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:45.045399 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:45.045302 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" event={"ID":"311cc01e-312f-4240-8ac4-57edace3ea5f","Type":"ContainerStarted","Data":"48d1bbdef114e145c0158a22a7806cb6ffce3b8f613722fa9341257c792ca505"} Jul 07 13:21:45.047176 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:21:45.047146 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cg46s\" is forbidden: User \"system:node:ip-10-0-136-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-215.ec2.internal' and this object" podUID="0d0b0181-ce71-4d74-be51-45992d8264b2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:45.047269 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:45.047244 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" event={"ID":"674be9eb-18ed-47f6-a0be-948f0fa0f80c","Type":"ContainerStarted","Data":"583f029eab86118940731ccd910645b9ddfb413af5f9e685e445ed21ede571ed"} Jul 07 13:21:45.077501 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:21:45.077447 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cg46s\" is forbidden: User \"system:node:ip-10-0-136-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-215.ec2.internal' and this object" podUID="0d0b0181-ce71-4d74-be51-45992d8264b2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:45.078400 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:45.077933 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b6t7b" podStartSLOduration=2.581016321 podStartE2EDuration="25.077915681s" totalImagesPullingTime="22.49689936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:21:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:21:45.0742806 +0000 UTC m=+462.752841932" watchObservedRunningTime="2026-07-07 13:21:45.077915681 +0000 UTC m=+462.756477013" Jul 07 13:21:45.119372 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:45.119227 2568 scope.go:117] "RemoveContainer" containerID="cb0e3738b4acfad27983f880cde4da33491cedd0d33c0751f3e07b1aa49516ba" Jul 07 13:21:45.119697 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:21:45.119663 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb0e3738b4acfad27983f880cde4da33491cedd0d33c0751f3e07b1aa49516ba\": container with ID starting with cb0e3738b4acfad27983f880cde4da33491cedd0d33c0751f3e07b1aa49516ba not found: ID does not exist" containerID="cb0e3738b4acfad27983f880cde4da33491cedd0d33c0751f3e07b1aa49516ba" Jul 07 13:21:45.119809 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:45.119712 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb0e3738b4acfad27983f880cde4da33491cedd0d33c0751f3e07b1aa49516ba"} err="failed to get container status \"cb0e3738b4acfad27983f880cde4da33491cedd0d33c0751f3e07b1aa49516ba\": rpc error: code = NotFound desc = could not find container \"cb0e3738b4acfad27983f880cde4da33491cedd0d33c0751f3e07b1aa49516ba\": container with ID starting with cb0e3738b4acfad27983f880cde4da33491cedd0d33c0751f3e07b1aa49516ba not found: ID does not exist" Jul 07 13:21:45.260568 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:45.260544 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5797dc48b7-8jsrb"] Jul 07 13:21:45.262776 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:21:45.262752 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4345a50e_990d_4242_a0f1_948b1257f288.slice/crio-a0c5fb89b4070a2ee47d7c44deeec6f3c6f8fa8c774975a7be06926abd29ba35 WatchSource:0}: Error finding container a0c5fb89b4070a2ee47d7c44deeec6f3c6f8fa8c774975a7be06926abd29ba35: Status 404 returned error can't find the container with id a0c5fb89b4070a2ee47d7c44deeec6f3c6f8fa8c774975a7be06926abd29ba35 Jul 07 13:21:46.054306 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:46.054270 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5797dc48b7-8jsrb" event={"ID":"4345a50e-990d-4242-a0f1-948b1257f288","Type":"ContainerStarted","Data":"377c02a1c0a9ca0659b3da9111eb8f288d1a83b9f481a3a41a106d7aec9df780"} Jul 07 13:21:46.054495 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:46.054327 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5797dc48b7-8jsrb" event={"ID":"4345a50e-990d-4242-a0f1-948b1257f288","Type":"ContainerStarted","Data":"a0c5fb89b4070a2ee47d7c44deeec6f3c6f8fa8c774975a7be06926abd29ba35"} Jul 07 13:21:46.056881 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:21:46.056853 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cg46s\" is forbidden: User \"system:node:ip-10-0-136-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-215.ec2.internal' and this object" podUID="0d0b0181-ce71-4d74-be51-45992d8264b2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cg46s" Jul 07 13:21:46.076929 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:46.076884 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5797dc48b7-8jsrb" podStartSLOduration=2.076869464 podStartE2EDuration="2.076869464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:21:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:21:46.075521983 +0000 UTC m=+463.754083315" watchObservedRunningTime="2026-07-07 13:21:46.076869464 +0000 UTC m=+463.755430794" Jul 07 13:21:50.076866 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:50.076826 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" event={"ID":"674be9eb-18ed-47f6-a0be-948f0fa0f80c","Type":"ContainerStarted","Data":"1654dc65d8fab21300abe9594fde890f7934525bc8b12df7e71329f45dfa97eb"} Jul 07 13:21:50.077240 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:50.076971 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" Jul 07 13:21:50.114979 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:50.114916 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" podStartSLOduration=14.816040532 podStartE2EDuration="20.114890016s" totalImagesPullingTime="5.298849484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:21:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:21:50.11164105 +0000 UTC m=+467.790202382" watchObservedRunningTime="2026-07-07 13:21:50.114890016 +0000 UTC m=+467.793451351" Jul 07 13:21:52.611814 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:52.611775 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:52.749483 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:52.749446 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:53.215624 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:53.215592 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:53.297847 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:53.297818 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:54.954819 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:54.954785 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:54.955290 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:54.954844 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:54.959410 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:54.959389 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:55.103379 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:55.103354 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5797dc48b7-8jsrb" Jul 07 13:21:55.523135 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:55.523100 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:55.659232 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:55.659196 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:57.065407 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:57.065374 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:57.229036 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:21:57.229002 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:01.076338 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:01.076311 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:01.083785 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:01.083752 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" Jul 07 13:22:01.230532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:01.230499 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:04.624472 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:04.624427 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:04.724213 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:04.724178 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:13.978203 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:13.978172 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt"] Jul 07 13:22:13.978793 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:13.978421 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" podUID="674be9eb-18ed-47f6-a0be-948f0fa0f80c" containerName="manager" containerID="cri-o://1654dc65d8fab21300abe9594fde890f7934525bc8b12df7e71329f45dfa97eb" gracePeriod=10 Jul 07 13:22:14.024624 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:22:14.024569 2568 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod674be9eb_18ed_47f6_a0be_948f0fa0f80c.slice/crio-1654dc65d8fab21300abe9594fde890f7934525bc8b12df7e71329f45dfa97eb.scope\": RecentStats: unable to find data in memory cache]" Jul 07 13:22:14.189691 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:14.189653 2568 generic.go:356] "Generic (PLEG): container finished" podID="674be9eb-18ed-47f6-a0be-948f0fa0f80c" containerID="1654dc65d8fab21300abe9594fde890f7934525bc8b12df7e71329f45dfa97eb" exitCode=0 Jul 07 13:22:14.189832 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:14.189728 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" event={"ID":"674be9eb-18ed-47f6-a0be-948f0fa0f80c","Type":"ContainerDied","Data":"1654dc65d8fab21300abe9594fde890f7934525bc8b12df7e71329f45dfa97eb"} Jul 07 13:22:14.720190 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:14.720167 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" Jul 07 13:22:14.849992 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:14.849901 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2qhj\" (UniqueName: \"kubernetes.io/projected/674be9eb-18ed-47f6-a0be-948f0fa0f80c-kube-api-access-w2qhj\") pod \"674be9eb-18ed-47f6-a0be-948f0fa0f80c\" (UID: \"674be9eb-18ed-47f6-a0be-948f0fa0f80c\") " Jul 07 13:22:14.849992 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:14.849990 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/674be9eb-18ed-47f6-a0be-948f0fa0f80c-extensions-socket-volume\") pod \"674be9eb-18ed-47f6-a0be-948f0fa0f80c\" (UID: \"674be9eb-18ed-47f6-a0be-948f0fa0f80c\") " Jul 07 13:22:14.850341 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:14.850317 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/674be9eb-18ed-47f6-a0be-948f0fa0f80c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "674be9eb-18ed-47f6-a0be-948f0fa0f80c" (UID: "674be9eb-18ed-47f6-a0be-948f0fa0f80c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:22:14.851983 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:14.851958 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/674be9eb-18ed-47f6-a0be-948f0fa0f80c-kube-api-access-w2qhj" (OuterVolumeSpecName: "kube-api-access-w2qhj") pod "674be9eb-18ed-47f6-a0be-948f0fa0f80c" (UID: "674be9eb-18ed-47f6-a0be-948f0fa0f80c"). InnerVolumeSpecName "kube-api-access-w2qhj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:22:14.951194 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:14.951163 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/674be9eb-18ed-47f6-a0be-948f0fa0f80c-extensions-socket-volume\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:22:14.951194 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:14.951190 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2qhj\" (UniqueName: \"kubernetes.io/projected/674be9eb-18ed-47f6-a0be-948f0fa0f80c-kube-api-access-w2qhj\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:22:15.195510 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:15.195486 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" Jul 07 13:22:15.195978 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:15.195528 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt" event={"ID":"674be9eb-18ed-47f6-a0be-948f0fa0f80c","Type":"ContainerDied","Data":"583f029eab86118940731ccd910645b9ddfb413af5f9e685e445ed21ede571ed"} Jul 07 13:22:15.195978 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:15.195619 2568 scope.go:117] "RemoveContainer" containerID="1654dc65d8fab21300abe9594fde890f7934525bc8b12df7e71329f45dfa97eb" Jul 07 13:22:15.218381 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:15.218352 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt"] Jul 07 13:22:15.226351 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:15.226324 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcxgt"] Jul 07 13:22:16.971464 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:16.971423 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="674be9eb-18ed-47f6-a0be-948f0fa0f80c" path="/var/lib/kubelet/pods/674be9eb-18ed-47f6-a0be-948f0fa0f80c/volumes" Jul 07 13:22:17.856854 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.856819 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q"] Jul 07 13:22:17.857237 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.857223 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="674be9eb-18ed-47f6-a0be-948f0fa0f80c" containerName="manager" Jul 07 13:22:17.857303 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.857238 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="674be9eb-18ed-47f6-a0be-948f0fa0f80c" containerName="manager" Jul 07 13:22:17.857354 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.857307 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="674be9eb-18ed-47f6-a0be-948f0fa0f80c" containerName="manager" Jul 07 13:22:17.860264 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.860246 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:17.864323 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.864302 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-kqh7h\"" Jul 07 13:22:17.907504 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.907463 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:17.928993 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.928966 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q"] Jul 07 13:22:17.981501 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.981472 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:17.981891 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.981512 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:17.981891 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.981538 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6c35371-5324-4554-9831-222b6cde861a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:17.981891 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.981569 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:17.981891 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.981662 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6c35371-5324-4554-9831-222b6cde861a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:17.981891 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.981692 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6c35371-5324-4554-9831-222b6cde861a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:17.981891 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.981722 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:17.981891 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.981757 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:17.981891 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:17.981790 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whb2h\" (UniqueName: \"kubernetes.io/projected/c6c35371-5324-4554-9831-222b6cde861a-kube-api-access-whb2h\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.025054 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.025026 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:18.082592 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.082540 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.082777 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.082604 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whb2h\" (UniqueName: \"kubernetes.io/projected/c6c35371-5324-4554-9831-222b6cde861a-kube-api-access-whb2h\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.082777 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.082671 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.082777 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.082716 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.082777 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.082746 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6c35371-5324-4554-9831-222b6cde861a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.082777 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.082771 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.083035 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.082816 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6c35371-5324-4554-9831-222b6cde861a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.083035 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.082838 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6c35371-5324-4554-9831-222b6cde861a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.083035 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.082886 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.083176 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.083037 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.083176 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.083094 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.083278 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.083211 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.083278 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.083222 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.083655 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.083634 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6c35371-5324-4554-9831-222b6cde861a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.085495 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.085471 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6c35371-5324-4554-9831-222b6cde861a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.085644 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.085624 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6c35371-5324-4554-9831-222b6cde861a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.095689 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.095659 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6c35371-5324-4554-9831-222b6cde861a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.095906 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.095887 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whb2h\" (UniqueName: \"kubernetes.io/projected/c6c35371-5324-4554-9831-222b6cde861a-kube-api-access-whb2h\") pod \"maas-default-gateway-openshift-default-56654dcf7-j988q\" (UID: \"c6c35371-5324-4554-9831-222b6cde861a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.170789 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.170769 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:18.307311 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.307287 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q"] Jul 07 13:22:18.309536 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:22:18.309512 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6c35371_5324_4554_9831_222b6cde861a.slice/crio-fd56055b240c0f1adb016d9ae632dd17b8e68e13fa5792e6dc0feba048b6978a WatchSource:0}: Error finding container fd56055b240c0f1adb016d9ae632dd17b8e68e13fa5792e6dc0feba048b6978a: Status 404 returned error can't find the container with id fd56055b240c0f1adb016d9ae632dd17b8e68e13fa5792e6dc0feba048b6978a Jul 07 13:22:18.311610 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.311554 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:22:18.311699 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.311639 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:22:18.311699 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:18.311670 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:22:19.216132 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:19.216093 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" event={"ID":"c6c35371-5324-4554-9831-222b6cde861a","Type":"ContainerStarted","Data":"5c6634ab6461faf2bda4de48eaa1b9cc243fdd320a5750257e0c0fea7b4e2c46"} Jul 07 13:22:19.216474 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:19.216151 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" event={"ID":"c6c35371-5324-4554-9831-222b6cde861a","Type":"ContainerStarted","Data":"fd56055b240c0f1adb016d9ae632dd17b8e68e13fa5792e6dc0feba048b6978a"} Jul 07 13:22:19.724640 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:19.724610 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:20.171128 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.171105 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:20.176271 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.176248 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:20.206218 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.206173 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" podStartSLOduration=3.206157972 podStartE2EDuration="3.206157972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:22:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:19.255590922 +0000 UTC m=+496.934152250" watchObservedRunningTime="2026-07-07 13:22:20.206157972 +0000 UTC m=+497.884719302" Jul 07 13:22:20.221639 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.221612 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:20.222642 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.222623 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:22:20.360759 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.360722 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:20.361085 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.361067 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bt92l"] Jul 07 13:22:20.403500 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.403463 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bt92l"] Jul 07 13:22:20.403688 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.403570 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" Jul 07 13:22:20.406634 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.406609 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 13:22:20.452782 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.452695 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bt92l"] Jul 07 13:22:20.453182 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.453151 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:20.507741 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.507715 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cea7ee88-b145-4203-b735-8194b44d6985-config-file\") pod \"limitador-limitador-64c94fd64f-bt92l\" (UID: \"cea7ee88-b145-4203-b735-8194b44d6985\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" Jul 07 13:22:20.507874 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.507774 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhmhx\" (UniqueName: \"kubernetes.io/projected/cea7ee88-b145-4203-b735-8194b44d6985-kube-api-access-rhmhx\") pod \"limitador-limitador-64c94fd64f-bt92l\" (UID: \"cea7ee88-b145-4203-b735-8194b44d6985\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" Jul 07 13:22:20.548793 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.548765 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:20.608964 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.608934 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhmhx\" (UniqueName: \"kubernetes.io/projected/cea7ee88-b145-4203-b735-8194b44d6985-kube-api-access-rhmhx\") pod \"limitador-limitador-64c94fd64f-bt92l\" (UID: \"cea7ee88-b145-4203-b735-8194b44d6985\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" Jul 07 13:22:20.609109 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.609010 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cea7ee88-b145-4203-b735-8194b44d6985-config-file\") pod \"limitador-limitador-64c94fd64f-bt92l\" (UID: \"cea7ee88-b145-4203-b735-8194b44d6985\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" Jul 07 13:22:20.609560 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.609542 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cea7ee88-b145-4203-b735-8194b44d6985-config-file\") pod \"limitador-limitador-64c94fd64f-bt92l\" (UID: \"cea7ee88-b145-4203-b735-8194b44d6985\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" Jul 07 13:22:20.619224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.619196 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhmhx\" (UniqueName: \"kubernetes.io/projected/cea7ee88-b145-4203-b735-8194b44d6985-kube-api-access-rhmhx\") pod \"limitador-limitador-64c94fd64f-bt92l\" (UID: \"cea7ee88-b145-4203-b735-8194b44d6985\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" Jul 07 13:22:20.715326 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.715259 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" Jul 07 13:22:20.872772 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:20.872747 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bt92l"] Jul 07 13:22:20.874482 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:22:20.874444 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcea7ee88_b145_4203_b735_8194b44d6985.slice/crio-479d3b5c316f46ae4a435bcd5c28f4f3bca0381fa4a48e22e181644027d9d9e0 WatchSource:0}: Error finding container 479d3b5c316f46ae4a435bcd5c28f4f3bca0381fa4a48e22e181644027d9d9e0: Status 404 returned error can't find the container with id 479d3b5c316f46ae4a435bcd5c28f4f3bca0381fa4a48e22e181644027d9d9e0 Jul 07 13:22:21.227240 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:21.227206 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" event={"ID":"cea7ee88-b145-4203-b735-8194b44d6985","Type":"ContainerStarted","Data":"479d3b5c316f46ae4a435bcd5c28f4f3bca0381fa4a48e22e181644027d9d9e0"} Jul 07 13:22:21.288359 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:21.288332 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:21.401914 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:21.401884 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:24.246920 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:24.246885 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" event={"ID":"cea7ee88-b145-4203-b735-8194b44d6985","Type":"ContainerStarted","Data":"e5bdf78f3d37edc466b2717223ec35850c5620b5c74b348f7a421d4607d536fd"} Jul 07 13:22:24.247296 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:24.246959 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" Jul 07 13:22:24.268792 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:24.268745 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" podStartSLOduration=1.714895351 podStartE2EDuration="4.268729864s" totalImagesPullingTime="2.553834513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:22:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:24.2661264 +0000 UTC m=+501.944687734" watchObservedRunningTime="2026-07-07 13:22:24.268729864 +0000 UTC m=+501.947291194" Jul 07 13:22:35.252447 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:35.252416 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" Jul 07 13:22:37.724590 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:37.724553 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bt92l"] Jul 07 13:22:37.725165 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:37.724753 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" podUID="cea7ee88-b145-4203-b735-8194b44d6985" containerName="limitador" containerID="cri-o://e5bdf78f3d37edc466b2717223ec35850c5620b5c74b348f7a421d4607d536fd" gracePeriod=30 Jul 07 13:22:38.312862 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:38.312772 2568 generic.go:356] "Generic (PLEG): container finished" podID="cea7ee88-b145-4203-b735-8194b44d6985" containerID="e5bdf78f3d37edc466b2717223ec35850c5620b5c74b348f7a421d4607d536fd" exitCode=0 Jul 07 13:22:38.312862 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:38.312844 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" event={"ID":"cea7ee88-b145-4203-b735-8194b44d6985","Type":"ContainerDied","Data":"e5bdf78f3d37edc466b2717223ec35850c5620b5c74b348f7a421d4607d536fd"} Jul 07 13:22:38.668727 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:38.668704 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" Jul 07 13:22:38.798382 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:38.798343 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cea7ee88-b145-4203-b735-8194b44d6985-config-file\") pod \"cea7ee88-b145-4203-b735-8194b44d6985\" (UID: \"cea7ee88-b145-4203-b735-8194b44d6985\") " Jul 07 13:22:38.798382 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:38.798388 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhmhx\" (UniqueName: \"kubernetes.io/projected/cea7ee88-b145-4203-b735-8194b44d6985-kube-api-access-rhmhx\") pod \"cea7ee88-b145-4203-b735-8194b44d6985\" (UID: \"cea7ee88-b145-4203-b735-8194b44d6985\") " Jul 07 13:22:38.798843 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:38.798750 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cea7ee88-b145-4203-b735-8194b44d6985-config-file" (OuterVolumeSpecName: "config-file") pod "cea7ee88-b145-4203-b735-8194b44d6985" (UID: "cea7ee88-b145-4203-b735-8194b44d6985"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:22:38.800465 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:38.800439 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cea7ee88-b145-4203-b735-8194b44d6985-kube-api-access-rhmhx" (OuterVolumeSpecName: "kube-api-access-rhmhx") pod "cea7ee88-b145-4203-b735-8194b44d6985" (UID: "cea7ee88-b145-4203-b735-8194b44d6985"). InnerVolumeSpecName "kube-api-access-rhmhx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:22:38.899950 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:38.899861 2568 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cea7ee88-b145-4203-b735-8194b44d6985-config-file\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:22:38.899950 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:38.899892 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhmhx\" (UniqueName: \"kubernetes.io/projected/cea7ee88-b145-4203-b735-8194b44d6985-kube-api-access-rhmhx\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:22:39.318758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:39.318731 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" Jul 07 13:22:39.318758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:39.318740 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bt92l" event={"ID":"cea7ee88-b145-4203-b735-8194b44d6985","Type":"ContainerDied","Data":"479d3b5c316f46ae4a435bcd5c28f4f3bca0381fa4a48e22e181644027d9d9e0"} Jul 07 13:22:39.318991 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:39.318797 2568 scope.go:117] "RemoveContainer" containerID="e5bdf78f3d37edc466b2717223ec35850c5620b5c74b348f7a421d4607d536fd" Jul 07 13:22:39.350099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:39.350068 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bt92l"] Jul 07 13:22:39.354816 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:39.354791 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bt92l"] Jul 07 13:22:40.970493 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:40.970460 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cea7ee88-b145-4203-b735-8194b44d6985" path="/var/lib/kubelet/pods/cea7ee88-b145-4203-b735-8194b44d6985/volumes" Jul 07 13:22:41.884587 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:41.884546 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-qkhmw"] Jul 07 13:22:41.884954 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:41.884939 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea7ee88-b145-4203-b735-8194b44d6985" containerName="limitador" Jul 07 13:22:41.884954 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:41.884955 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea7ee88-b145-4203-b735-8194b44d6985" containerName="limitador" Jul 07 13:22:41.885079 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:41.885033 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea7ee88-b145-4203-b735-8194b44d6985" containerName="limitador" Jul 07 13:22:41.888096 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:41.888075 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-qkhmw" Jul 07 13:22:41.890626 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:41.890556 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 07 13:22:41.890748 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:41.890702 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8ptwd\"" Jul 07 13:22:41.896049 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:41.896029 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-qkhmw"] Jul 07 13:22:41.927345 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:41.927323 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:42.030759 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:42.030722 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9gsm\" (UniqueName: \"kubernetes.io/projected/149d4a33-2ae9-405b-89b8-ee816e556ebf-kube-api-access-l9gsm\") pod \"postgres-69fb4594cf-qkhmw\" (UID: \"149d4a33-2ae9-405b-89b8-ee816e556ebf\") " pod="opendatahub/postgres-69fb4594cf-qkhmw" Jul 07 13:22:42.031203 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:42.030790 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/149d4a33-2ae9-405b-89b8-ee816e556ebf-data\") pod \"postgres-69fb4594cf-qkhmw\" (UID: \"149d4a33-2ae9-405b-89b8-ee816e556ebf\") " pod="opendatahub/postgres-69fb4594cf-qkhmw" Jul 07 13:22:42.132158 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:42.132121 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9gsm\" (UniqueName: \"kubernetes.io/projected/149d4a33-2ae9-405b-89b8-ee816e556ebf-kube-api-access-l9gsm\") pod \"postgres-69fb4594cf-qkhmw\" (UID: \"149d4a33-2ae9-405b-89b8-ee816e556ebf\") " pod="opendatahub/postgres-69fb4594cf-qkhmw" Jul 07 13:22:42.132158 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:42.132161 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/149d4a33-2ae9-405b-89b8-ee816e556ebf-data\") pod \"postgres-69fb4594cf-qkhmw\" (UID: \"149d4a33-2ae9-405b-89b8-ee816e556ebf\") " pod="opendatahub/postgres-69fb4594cf-qkhmw" Jul 07 13:22:42.132625 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:42.132600 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/149d4a33-2ae9-405b-89b8-ee816e556ebf-data\") pod \"postgres-69fb4594cf-qkhmw\" (UID: \"149d4a33-2ae9-405b-89b8-ee816e556ebf\") " pod="opendatahub/postgres-69fb4594cf-qkhmw" Jul 07 13:22:42.145080 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:42.145019 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9gsm\" (UniqueName: \"kubernetes.io/projected/149d4a33-2ae9-405b-89b8-ee816e556ebf-kube-api-access-l9gsm\") pod \"postgres-69fb4594cf-qkhmw\" (UID: \"149d4a33-2ae9-405b-89b8-ee816e556ebf\") " pod="opendatahub/postgres-69fb4594cf-qkhmw" Jul 07 13:22:42.199564 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:42.199530 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-qkhmw" Jul 07 13:22:42.331693 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:42.331660 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-qkhmw"] Jul 07 13:22:43.347161 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:43.347045 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-qkhmw" event={"ID":"149d4a33-2ae9-405b-89b8-ee816e556ebf","Type":"ContainerStarted","Data":"a769f0bb8ca4556f6c462ef32be94530e50713f796ed5e3609bd6383a23c7294"} Jul 07 13:22:47.370117 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:47.370016 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-qkhmw" event={"ID":"149d4a33-2ae9-405b-89b8-ee816e556ebf","Type":"ContainerStarted","Data":"15de8787b9f7a1e3f1c80b7b2fb100da2d54a91c73fb09fe3bdf186cc65753cf"} Jul 07 13:22:47.370117 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:47.370074 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-qkhmw" Jul 07 13:22:47.397711 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:47.397656 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-qkhmw" podStartSLOduration=1.706326784 podStartE2EDuration="6.397640778s" totalImagesPullingTime="4.691313994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:22:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:47.393465059 +0000 UTC m=+525.072026391" watchObservedRunningTime="2026-07-07 13:22:47.397640778 +0000 UTC m=+525.076202109" Jul 07 13:22:53.405193 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:53.405160 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-qkhmw" Jul 07 13:22:53.791190 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:53.791136 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:53.923492 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:22:53.923461 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:23:00.710809 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:00.710725 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:23:00.751713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:00.751680 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7864899994-ctf56"] Jul 07 13:23:00.770485 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:00.770459 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7864899994-ctf56" Jul 07 13:23:00.773920 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:00.773896 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7864899994-ctf56"] Jul 07 13:23:00.775415 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:00.775369 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 13:23:00.775544 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:00.775473 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pzvhp\"" Jul 07 13:23:00.775620 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:00.775592 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 13:23:00.869297 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:00.869263 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:23:00.918554 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:00.918520 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db3ea6be-fd23-41b2-b93d-16243b37a332-webhook-cert\") pod \"maas-controller-7864899994-ctf56\" (UID: \"db3ea6be-fd23-41b2-b93d-16243b37a332\") " pod="opendatahub/maas-controller-7864899994-ctf56" Jul 07 13:23:00.918740 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:00.918597 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrzsh\" (UniqueName: \"kubernetes.io/projected/db3ea6be-fd23-41b2-b93d-16243b37a332-kube-api-access-zrzsh\") pod \"maas-controller-7864899994-ctf56\" (UID: \"db3ea6be-fd23-41b2-b93d-16243b37a332\") " pod="opendatahub/maas-controller-7864899994-ctf56" Jul 07 13:23:00.971897 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:00.971826 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-777d69b966-mp8xq"] Jul 07 13:23:00.974255 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:00.974239 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-777d69b966-mp8xq" Jul 07 13:23:00.989326 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:00.989303 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-777d69b966-mp8xq"] Jul 07 13:23:00.991050 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:00.991027 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:23:01.019516 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.019491 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrzsh\" (UniqueName: \"kubernetes.io/projected/db3ea6be-fd23-41b2-b93d-16243b37a332-kube-api-access-zrzsh\") pod \"maas-controller-7864899994-ctf56\" (UID: \"db3ea6be-fd23-41b2-b93d-16243b37a332\") " pod="opendatahub/maas-controller-7864899994-ctf56" Jul 07 13:23:01.019684 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.019598 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db3ea6be-fd23-41b2-b93d-16243b37a332-webhook-cert\") pod \"maas-controller-7864899994-ctf56\" (UID: \"db3ea6be-fd23-41b2-b93d-16243b37a332\") " pod="opendatahub/maas-controller-7864899994-ctf56" Jul 07 13:23:01.021904 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.021884 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db3ea6be-fd23-41b2-b93d-16243b37a332-webhook-cert\") pod \"maas-controller-7864899994-ctf56\" (UID: \"db3ea6be-fd23-41b2-b93d-16243b37a332\") " pod="opendatahub/maas-controller-7864899994-ctf56" Jul 07 13:23:01.031285 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.031268 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrzsh\" (UniqueName: \"kubernetes.io/projected/db3ea6be-fd23-41b2-b93d-16243b37a332-kube-api-access-zrzsh\") pod \"maas-controller-7864899994-ctf56\" (UID: \"db3ea6be-fd23-41b2-b93d-16243b37a332\") " pod="opendatahub/maas-controller-7864899994-ctf56" Jul 07 13:23:01.081785 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.081753 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7864899994-ctf56" Jul 07 13:23:01.120567 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.120526 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf6965e-2cc6-42b0-8e16-45c8838cfab6-webhook-cert\") pod \"maas-controller-777d69b966-mp8xq\" (UID: \"8bf6965e-2cc6-42b0-8e16-45c8838cfab6\") " pod="opendatahub/maas-controller-777d69b966-mp8xq" Jul 07 13:23:01.120748 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.120651 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtzzx\" (UniqueName: \"kubernetes.io/projected/8bf6965e-2cc6-42b0-8e16-45c8838cfab6-kube-api-access-mtzzx\") pod \"maas-controller-777d69b966-mp8xq\" (UID: \"8bf6965e-2cc6-42b0-8e16-45c8838cfab6\") " pod="opendatahub/maas-controller-777d69b966-mp8xq" Jul 07 13:23:01.217110 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.217088 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7864899994-ctf56"] Jul 07 13:23:01.219341 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:23:01.219300 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb3ea6be_fd23_41b2_b93d_16243b37a332.slice/crio-b3965842b9fcac2926560a66aa6ca44d2a4afba2e83c915e8aea89245d7d73fc WatchSource:0}: Error finding container b3965842b9fcac2926560a66aa6ca44d2a4afba2e83c915e8aea89245d7d73fc: Status 404 returned error can't find the container with id b3965842b9fcac2926560a66aa6ca44d2a4afba2e83c915e8aea89245d7d73fc Jul 07 13:23:01.221203 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.221178 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtzzx\" (UniqueName: \"kubernetes.io/projected/8bf6965e-2cc6-42b0-8e16-45c8838cfab6-kube-api-access-mtzzx\") pod \"maas-controller-777d69b966-mp8xq\" (UID: \"8bf6965e-2cc6-42b0-8e16-45c8838cfab6\") " pod="opendatahub/maas-controller-777d69b966-mp8xq" Jul 07 13:23:01.221322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.221277 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf6965e-2cc6-42b0-8e16-45c8838cfab6-webhook-cert\") pod \"maas-controller-777d69b966-mp8xq\" (UID: \"8bf6965e-2cc6-42b0-8e16-45c8838cfab6\") " pod="opendatahub/maas-controller-777d69b966-mp8xq" Jul 07 13:23:01.223813 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.223781 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf6965e-2cc6-42b0-8e16-45c8838cfab6-webhook-cert\") pod \"maas-controller-777d69b966-mp8xq\" (UID: \"8bf6965e-2cc6-42b0-8e16-45c8838cfab6\") " pod="opendatahub/maas-controller-777d69b966-mp8xq" Jul 07 13:23:01.240693 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.240673 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtzzx\" (UniqueName: \"kubernetes.io/projected/8bf6965e-2cc6-42b0-8e16-45c8838cfab6-kube-api-access-mtzzx\") pod \"maas-controller-777d69b966-mp8xq\" (UID: \"8bf6965e-2cc6-42b0-8e16-45c8838cfab6\") " pod="opendatahub/maas-controller-777d69b966-mp8xq" Jul 07 13:23:01.284607 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.284565 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-777d69b966-mp8xq" Jul 07 13:23:01.437634 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.437604 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7864899994-ctf56" event={"ID":"db3ea6be-fd23-41b2-b93d-16243b37a332","Type":"ContainerStarted","Data":"b3965842b9fcac2926560a66aa6ca44d2a4afba2e83c915e8aea89245d7d73fc"} Jul 07 13:23:01.444209 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:01.444186 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-777d69b966-mp8xq"] Jul 07 13:23:01.445949 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:23:01.445925 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf6965e_2cc6_42b0_8e16_45c8838cfab6.slice/crio-187b833943667c14d21801353b2ea62c1ff0d85ffd268b1139c3637f80e53ef9 WatchSource:0}: Error finding container 187b833943667c14d21801353b2ea62c1ff0d85ffd268b1139c3637f80e53ef9: Status 404 returned error can't find the container with id 187b833943667c14d21801353b2ea62c1ff0d85ffd268b1139c3637f80e53ef9 Jul 07 13:23:02.445441 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:02.445384 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-777d69b966-mp8xq" event={"ID":"8bf6965e-2cc6-42b0-8e16-45c8838cfab6","Type":"ContainerStarted","Data":"187b833943667c14d21801353b2ea62c1ff0d85ffd268b1139c3637f80e53ef9"} Jul 07 13:23:03.500331 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:03.500309 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 13:23:03.738400 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:03.738361 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:23:03.827282 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:03.827198 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:23:04.457756 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:04.457717 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7864899994-ctf56" event={"ID":"db3ea6be-fd23-41b2-b93d-16243b37a332","Type":"ContainerStarted","Data":"557ae6a1aae47a05850b114cad2864ffcf6503ac1d3727c1cc8642b74afda827"} Jul 07 13:23:04.457951 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:04.457885 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7864899994-ctf56" Jul 07 13:23:04.459170 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:04.459144 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-777d69b966-mp8xq" event={"ID":"8bf6965e-2cc6-42b0-8e16-45c8838cfab6","Type":"ContainerStarted","Data":"26eac01b1e64faae5ff5b4e77d17c6431216e23a10d9ba38933ca1adb293d6a9"} Jul 07 13:23:04.459237 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:04.459218 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-777d69b966-mp8xq" Jul 07 13:23:04.486353 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:04.486308 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7864899994-ctf56" podStartSLOduration=2.209551735 podStartE2EDuration="4.486294736s" totalImagesPullingTime="2.276743001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:23:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:23:04.485052457 +0000 UTC m=+542.163613788" watchObservedRunningTime="2026-07-07 13:23:04.486294736 +0000 UTC m=+542.164856098" Jul 07 13:23:15.470487 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:15.470455 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7864899994-ctf56" Jul 07 13:23:15.470936 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:15.470533 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-777d69b966-mp8xq" Jul 07 13:23:15.490026 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:15.489981 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-777d69b966-mp8xq" podStartSLOduration=13.433805666 podStartE2EDuration="15.489961384s" totalImagesPullingTime="2.056155718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:23:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:23:04.544547407 +0000 UTC m=+542.223108737" watchObservedRunningTime="2026-07-07 13:23:15.489961384 +0000 UTC m=+553.168522721" Jul 07 13:23:15.545049 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:15.544976 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7864899994-ctf56"] Jul 07 13:23:15.545377 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:15.545232 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7864899994-ctf56" podUID="db3ea6be-fd23-41b2-b93d-16243b37a332" containerName="manager" containerID="cri-o://557ae6a1aae47a05850b114cad2864ffcf6503ac1d3727c1cc8642b74afda827" gracePeriod=30 Jul 07 13:23:15.791481 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:15.791460 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7864899994-ctf56" Jul 07 13:23:15.868757 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:15.868720 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrzsh\" (UniqueName: \"kubernetes.io/projected/db3ea6be-fd23-41b2-b93d-16243b37a332-kube-api-access-zrzsh\") pod \"db3ea6be-fd23-41b2-b93d-16243b37a332\" (UID: \"db3ea6be-fd23-41b2-b93d-16243b37a332\") " Jul 07 13:23:15.868944 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:15.868785 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db3ea6be-fd23-41b2-b93d-16243b37a332-webhook-cert\") pod \"db3ea6be-fd23-41b2-b93d-16243b37a332\" (UID: \"db3ea6be-fd23-41b2-b93d-16243b37a332\") " Jul 07 13:23:15.870886 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:15.870854 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db3ea6be-fd23-41b2-b93d-16243b37a332-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "db3ea6be-fd23-41b2-b93d-16243b37a332" (UID: "db3ea6be-fd23-41b2-b93d-16243b37a332"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:23:15.871023 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:15.870913 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db3ea6be-fd23-41b2-b93d-16243b37a332-kube-api-access-zrzsh" (OuterVolumeSpecName: "kube-api-access-zrzsh") pod "db3ea6be-fd23-41b2-b93d-16243b37a332" (UID: "db3ea6be-fd23-41b2-b93d-16243b37a332"). InnerVolumeSpecName "kube-api-access-zrzsh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:23:15.969799 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:15.969766 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrzsh\" (UniqueName: \"kubernetes.io/projected/db3ea6be-fd23-41b2-b93d-16243b37a332-kube-api-access-zrzsh\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:23:15.969799 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:15.969795 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db3ea6be-fd23-41b2-b93d-16243b37a332-webhook-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:23:16.526877 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:16.526842 2568 generic.go:356] "Generic (PLEG): container finished" podID="db3ea6be-fd23-41b2-b93d-16243b37a332" containerID="557ae6a1aae47a05850b114cad2864ffcf6503ac1d3727c1cc8642b74afda827" exitCode=0 Jul 07 13:23:16.527273 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:16.526889 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7864899994-ctf56" Jul 07 13:23:16.527273 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:16.526928 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7864899994-ctf56" event={"ID":"db3ea6be-fd23-41b2-b93d-16243b37a332","Type":"ContainerDied","Data":"557ae6a1aae47a05850b114cad2864ffcf6503ac1d3727c1cc8642b74afda827"} Jul 07 13:23:16.527273 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:16.526982 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7864899994-ctf56" event={"ID":"db3ea6be-fd23-41b2-b93d-16243b37a332","Type":"ContainerDied","Data":"b3965842b9fcac2926560a66aa6ca44d2a4afba2e83c915e8aea89245d7d73fc"} Jul 07 13:23:16.527273 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:16.527005 2568 scope.go:117] "RemoveContainer" containerID="557ae6a1aae47a05850b114cad2864ffcf6503ac1d3727c1cc8642b74afda827" Jul 07 13:23:16.535860 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:16.535841 2568 scope.go:117] "RemoveContainer" containerID="557ae6a1aae47a05850b114cad2864ffcf6503ac1d3727c1cc8642b74afda827" Jul 07 13:23:16.536115 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:23:16.536100 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"557ae6a1aae47a05850b114cad2864ffcf6503ac1d3727c1cc8642b74afda827\": container with ID starting with 557ae6a1aae47a05850b114cad2864ffcf6503ac1d3727c1cc8642b74afda827 not found: ID does not exist" containerID="557ae6a1aae47a05850b114cad2864ffcf6503ac1d3727c1cc8642b74afda827" Jul 07 13:23:16.536159 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:16.536122 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"557ae6a1aae47a05850b114cad2864ffcf6503ac1d3727c1cc8642b74afda827"} err="failed to get container status \"557ae6a1aae47a05850b114cad2864ffcf6503ac1d3727c1cc8642b74afda827\": rpc error: code = NotFound desc = could not find container \"557ae6a1aae47a05850b114cad2864ffcf6503ac1d3727c1cc8642b74afda827\": container with ID starting with 557ae6a1aae47a05850b114cad2864ffcf6503ac1d3727c1cc8642b74afda827 not found: ID does not exist" Jul 07 13:23:16.551056 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:16.551037 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7864899994-ctf56"] Jul 07 13:23:16.555193 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:16.555168 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7864899994-ctf56"] Jul 07 13:23:16.971852 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:16.971817 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db3ea6be-fd23-41b2-b93d-16243b37a332" path="/var/lib/kubelet/pods/db3ea6be-fd23-41b2-b93d-16243b37a332/volumes" Jul 07 13:23:36.154243 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.154209 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:23:36.679815 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.679776 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:23:36.732757 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.732726 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-65fc57dc76-ppprg"] Jul 07 13:23:36.733133 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.733121 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db3ea6be-fd23-41b2-b93d-16243b37a332" containerName="manager" Jul 07 13:23:36.733171 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.733134 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="db3ea6be-fd23-41b2-b93d-16243b37a332" containerName="manager" Jul 07 13:23:36.733221 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.733213 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="db3ea6be-fd23-41b2-b93d-16243b37a332" containerName="manager" Jul 07 13:23:36.735459 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.735444 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-65fc57dc76-ppprg" Jul 07 13:23:36.742514 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.742495 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 07 13:23:36.742630 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.742525 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-pxjh4\"" Jul 07 13:23:36.765559 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.765531 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-65fc57dc76-ppprg"] Jul 07 13:23:36.867022 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.866991 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrb29\" (UniqueName: \"kubernetes.io/projected/fc812a6b-8c7e-4b2a-9e82-78aa52315c96-kube-api-access-rrb29\") pod \"maas-api-65fc57dc76-ppprg\" (UID: \"fc812a6b-8c7e-4b2a-9e82-78aa52315c96\") " pod="opendatahub/maas-api-65fc57dc76-ppprg" Jul 07 13:23:36.867204 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.867046 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc812a6b-8c7e-4b2a-9e82-78aa52315c96-openshift-service-ca\") pod \"maas-api-65fc57dc76-ppprg\" (UID: \"fc812a6b-8c7e-4b2a-9e82-78aa52315c96\") " pod="opendatahub/maas-api-65fc57dc76-ppprg" Jul 07 13:23:36.867204 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.867124 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc812a6b-8c7e-4b2a-9e82-78aa52315c96-maas-api-tls\") pod \"maas-api-65fc57dc76-ppprg\" (UID: \"fc812a6b-8c7e-4b2a-9e82-78aa52315c96\") " pod="opendatahub/maas-api-65fc57dc76-ppprg" Jul 07 13:23:36.914135 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.914105 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:23:36.968241 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.968160 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrb29\" (UniqueName: \"kubernetes.io/projected/fc812a6b-8c7e-4b2a-9e82-78aa52315c96-kube-api-access-rrb29\") pod \"maas-api-65fc57dc76-ppprg\" (UID: \"fc812a6b-8c7e-4b2a-9e82-78aa52315c96\") " pod="opendatahub/maas-api-65fc57dc76-ppprg" Jul 07 13:23:36.968241 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.968207 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc812a6b-8c7e-4b2a-9e82-78aa52315c96-openshift-service-ca\") pod \"maas-api-65fc57dc76-ppprg\" (UID: \"fc812a6b-8c7e-4b2a-9e82-78aa52315c96\") " pod="opendatahub/maas-api-65fc57dc76-ppprg" Jul 07 13:23:36.968241 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.968226 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc812a6b-8c7e-4b2a-9e82-78aa52315c96-maas-api-tls\") pod \"maas-api-65fc57dc76-ppprg\" (UID: \"fc812a6b-8c7e-4b2a-9e82-78aa52315c96\") " pod="opendatahub/maas-api-65fc57dc76-ppprg" Jul 07 13:23:36.968930 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.968910 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc812a6b-8c7e-4b2a-9e82-78aa52315c96-openshift-service-ca\") pod \"maas-api-65fc57dc76-ppprg\" (UID: \"fc812a6b-8c7e-4b2a-9e82-78aa52315c96\") " pod="opendatahub/maas-api-65fc57dc76-ppprg" Jul 07 13:23:36.970661 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.970639 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc812a6b-8c7e-4b2a-9e82-78aa52315c96-maas-api-tls\") pod \"maas-api-65fc57dc76-ppprg\" (UID: \"fc812a6b-8c7e-4b2a-9e82-78aa52315c96\") " pod="opendatahub/maas-api-65fc57dc76-ppprg" Jul 07 13:23:36.983340 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:36.983316 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrb29\" (UniqueName: \"kubernetes.io/projected/fc812a6b-8c7e-4b2a-9e82-78aa52315c96-kube-api-access-rrb29\") pod \"maas-api-65fc57dc76-ppprg\" (UID: \"fc812a6b-8c7e-4b2a-9e82-78aa52315c96\") " pod="opendatahub/maas-api-65fc57dc76-ppprg" Jul 07 13:23:37.044994 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.044962 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-65fc57dc76-ppprg" Jul 07 13:23:37.184073 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.184048 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-65fc57dc76-ppprg"] Jul 07 13:23:37.185905 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:23:37.185876 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc812a6b_8c7e_4b2a_9e82_78aa52315c96.slice/crio-5b0383791ffa3887d045fe8862fd9041be42130c3769c9ca59cc3a5ec5f46ac7 WatchSource:0}: Error finding container 5b0383791ffa3887d045fe8862fd9041be42130c3769c9ca59cc3a5ec5f46ac7: Status 404 returned error can't find the container with id 5b0383791ffa3887d045fe8862fd9041be42130c3769c9ca59cc3a5ec5f46ac7 Jul 07 13:23:37.387283 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.387212 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:23:37.529094 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.529052 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:23:37.623186 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.623156 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-z95v9"] Jul 07 13:23:37.627019 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.626998 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-z95v9" Jul 07 13:23:37.630027 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.629990 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 13:23:37.630135 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.630082 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-psj75\"" Jul 07 13:23:37.633427 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.633396 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-65fc57dc76-ppprg" event={"ID":"fc812a6b-8c7e-4b2a-9e82-78aa52315c96","Type":"ContainerStarted","Data":"5b0383791ffa3887d045fe8862fd9041be42130c3769c9ca59cc3a5ec5f46ac7"} Jul 07 13:23:37.651425 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.651359 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-z95v9"] Jul 07 13:23:37.679431 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.679398 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-llz2n"] Jul 07 13:23:37.682365 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.682337 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-llz2n" Jul 07 13:23:37.695345 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.695322 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-llz2n"] Jul 07 13:23:37.776186 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.776150 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7sc7\" (UniqueName: \"kubernetes.io/projected/0a99dba7-0297-4447-9928-91b8e05bd87b-kube-api-access-s7sc7\") pod \"payload-processing-58f86c7798-llz2n\" (UID: \"0a99dba7-0297-4447-9928-91b8e05bd87b\") " pod="openshift-ingress/payload-processing-58f86c7798-llz2n" Jul 07 13:23:37.776339 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.776208 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a99dba7-0297-4447-9928-91b8e05bd87b-plugins-config-volume\") pod \"payload-processing-58f86c7798-llz2n\" (UID: \"0a99dba7-0297-4447-9928-91b8e05bd87b\") " pod="openshift-ingress/payload-processing-58f86c7798-llz2n" Jul 07 13:23:37.776339 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.776255 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b169cfbe-f84a-483e-8659-d2d720666b7b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-z95v9\" (UID: \"b169cfbe-f84a-483e-8659-d2d720666b7b\") " pod="openshift-ingress/payload-pre-processing-7964457696-z95v9" Jul 07 13:23:37.776517 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.776375 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nrkx\" (UniqueName: \"kubernetes.io/projected/b169cfbe-f84a-483e-8659-d2d720666b7b-kube-api-access-9nrkx\") pod \"payload-pre-processing-7964457696-z95v9\" (UID: \"b169cfbe-f84a-483e-8659-d2d720666b7b\") " pod="openshift-ingress/payload-pre-processing-7964457696-z95v9" Jul 07 13:23:37.877641 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.877609 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7sc7\" (UniqueName: \"kubernetes.io/projected/0a99dba7-0297-4447-9928-91b8e05bd87b-kube-api-access-s7sc7\") pod \"payload-processing-58f86c7798-llz2n\" (UID: \"0a99dba7-0297-4447-9928-91b8e05bd87b\") " pod="openshift-ingress/payload-processing-58f86c7798-llz2n" Jul 07 13:23:37.877824 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.877652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a99dba7-0297-4447-9928-91b8e05bd87b-plugins-config-volume\") pod \"payload-processing-58f86c7798-llz2n\" (UID: \"0a99dba7-0297-4447-9928-91b8e05bd87b\") " pod="openshift-ingress/payload-processing-58f86c7798-llz2n" Jul 07 13:23:37.877824 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.877680 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b169cfbe-f84a-483e-8659-d2d720666b7b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-z95v9\" (UID: \"b169cfbe-f84a-483e-8659-d2d720666b7b\") " pod="openshift-ingress/payload-pre-processing-7964457696-z95v9" Jul 07 13:23:37.877824 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.877743 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nrkx\" (UniqueName: \"kubernetes.io/projected/b169cfbe-f84a-483e-8659-d2d720666b7b-kube-api-access-9nrkx\") pod \"payload-pre-processing-7964457696-z95v9\" (UID: \"b169cfbe-f84a-483e-8659-d2d720666b7b\") " pod="openshift-ingress/payload-pre-processing-7964457696-z95v9" Jul 07 13:23:37.878401 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.878377 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b169cfbe-f84a-483e-8659-d2d720666b7b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-z95v9\" (UID: \"b169cfbe-f84a-483e-8659-d2d720666b7b\") " pod="openshift-ingress/payload-pre-processing-7964457696-z95v9" Jul 07 13:23:37.878498 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.878416 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a99dba7-0297-4447-9928-91b8e05bd87b-plugins-config-volume\") pod \"payload-processing-58f86c7798-llz2n\" (UID: \"0a99dba7-0297-4447-9928-91b8e05bd87b\") " pod="openshift-ingress/payload-processing-58f86c7798-llz2n" Jul 07 13:23:37.887615 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.887569 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nrkx\" (UniqueName: \"kubernetes.io/projected/b169cfbe-f84a-483e-8659-d2d720666b7b-kube-api-access-9nrkx\") pod \"payload-pre-processing-7964457696-z95v9\" (UID: \"b169cfbe-f84a-483e-8659-d2d720666b7b\") " pod="openshift-ingress/payload-pre-processing-7964457696-z95v9" Jul 07 13:23:37.887707 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.887613 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7sc7\" (UniqueName: \"kubernetes.io/projected/0a99dba7-0297-4447-9928-91b8e05bd87b-kube-api-access-s7sc7\") pod \"payload-processing-58f86c7798-llz2n\" (UID: \"0a99dba7-0297-4447-9928-91b8e05bd87b\") " pod="openshift-ingress/payload-processing-58f86c7798-llz2n" Jul 07 13:23:37.937779 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.937748 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-z95v9" Jul 07 13:23:37.996600 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:37.996542 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-llz2n" Jul 07 13:23:38.126858 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:38.126764 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-z95v9"] Jul 07 13:23:38.129690 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:23:38.129661 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb169cfbe_f84a_483e_8659_d2d720666b7b.slice/crio-9758edfda911fe5f7e92eaaf1f3e03e8fd4446e7211aa9bd27b77fd93980d88b WatchSource:0}: Error finding container 9758edfda911fe5f7e92eaaf1f3e03e8fd4446e7211aa9bd27b77fd93980d88b: Status 404 returned error can't find the container with id 9758edfda911fe5f7e92eaaf1f3e03e8fd4446e7211aa9bd27b77fd93980d88b Jul 07 13:23:38.189809 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:38.189769 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-llz2n"] Jul 07 13:23:38.191768 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:23:38.191742 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a99dba7_0297_4447_9928_91b8e05bd87b.slice/crio-072fafc710e2a67878bb1196c801bbf70409ffa360e93d960c8cee2d0abe30c3 WatchSource:0}: Error finding container 072fafc710e2a67878bb1196c801bbf70409ffa360e93d960c8cee2d0abe30c3: Status 404 returned error can't find the container with id 072fafc710e2a67878bb1196c801bbf70409ffa360e93d960c8cee2d0abe30c3 Jul 07 13:23:38.660806 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:38.660707 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-z95v9" event={"ID":"b169cfbe-f84a-483e-8659-d2d720666b7b","Type":"ContainerStarted","Data":"9758edfda911fe5f7e92eaaf1f3e03e8fd4446e7211aa9bd27b77fd93980d88b"} Jul 07 13:23:38.662049 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:38.662019 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-llz2n" event={"ID":"0a99dba7-0297-4447-9928-91b8e05bd87b","Type":"ContainerStarted","Data":"072fafc710e2a67878bb1196c801bbf70409ffa360e93d960c8cee2d0abe30c3"} Jul 07 13:23:39.672404 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:39.672361 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-65fc57dc76-ppprg" event={"ID":"fc812a6b-8c7e-4b2a-9e82-78aa52315c96","Type":"ContainerStarted","Data":"6bfeed553a81d93cfab4ff0cebdd0a31135147d11e46cf83cdcab3796cc67121"} Jul 07 13:23:39.672884 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:39.672634 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-65fc57dc76-ppprg" Jul 07 13:23:39.698271 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:39.698207 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-65fc57dc76-ppprg" podStartSLOduration=1.9128465289999999 podStartE2EDuration="3.698181787s" totalImagesPullingTime="1.785335258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:23:39.696881755 +0000 UTC m=+577.375443088" watchObservedRunningTime="2026-07-07 13:23:39.698181787 +0000 UTC m=+577.376743117" Jul 07 13:23:40.678851 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:40.678816 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-z95v9" event={"ID":"b169cfbe-f84a-483e-8659-d2d720666b7b","Type":"ContainerStarted","Data":"c683e771d059c1ff959871faae9b0e763ca0ebac07201970b343831962e04c60"} Jul 07 13:23:40.679300 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:40.678977 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-z95v9" Jul 07 13:23:40.680473 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:40.680435 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-llz2n" event={"ID":"0a99dba7-0297-4447-9928-91b8e05bd87b","Type":"ContainerStarted","Data":"edb6e313994e00946eaf8e50e38246459fe32124a165bcf19b518e85312450ff"} Jul 07 13:23:40.680631 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:40.680605 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-llz2n" Jul 07 13:23:40.708645 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:40.708603 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-z95v9" podStartSLOduration=1.7327808679999999 podStartE2EDuration="3.708589283s" totalImagesPullingTime="1.975808415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:23:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:23:40.707619316 +0000 UTC m=+578.386180645" watchObservedRunningTime="2026-07-07 13:23:40.708589283 +0000 UTC m=+578.387150605" Jul 07 13:23:45.685938 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:45.685909 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-65fc57dc76-ppprg" Jul 07 13:23:45.707618 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:45.707563 2568 generic.go:356] "Generic (PLEG): container finished" podID="c6c35371-5324-4554-9831-222b6cde861a" containerID="5c6634ab6461faf2bda4de48eaa1b9cc243fdd320a5750257e0c0fea7b4e2c46" exitCode=0 Jul 07 13:23:45.707753 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:45.707638 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" event={"ID":"c6c35371-5324-4554-9831-222b6cde861a","Type":"ContainerDied","Data":"5c6634ab6461faf2bda4de48eaa1b9cc243fdd320a5750257e0c0fea7b4e2c46"} Jul 07 13:23:45.708141 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:45.708117 2568 scope.go:117] "RemoveContainer" containerID="5c6634ab6461faf2bda4de48eaa1b9cc243fdd320a5750257e0c0fea7b4e2c46" Jul 07 13:23:45.709300 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:45.709275 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:23:45.709358 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:45.709342 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:23:45.709393 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:45.709375 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:23:45.710025 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:45.709982 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-llz2n" podStartSLOduration=6.819435761 podStartE2EDuration="8.709965561s" totalImagesPullingTime="1.8905298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:23:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:23:40.753383011 +0000 UTC m=+578.431944342" watchObservedRunningTime="2026-07-07 13:23:45.709965561 +0000 UTC m=+583.388526893" Jul 07 13:23:46.715046 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:46.715010 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" event={"ID":"c6c35371-5324-4554-9831-222b6cde861a","Type":"ContainerStarted","Data":"c1c3fef9f543637ddfe6227a91ddb019a3846cce02f2360509f2c1b9921ae624"} Jul 07 13:23:47.171438 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:47.171400 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:23:47.172677 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:47.172647 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-j988q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 07 13:23:47.172820 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:47.172708 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" podUID="c6c35371-5324-4554-9831-222b6cde861a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 07 13:23:47.235041 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:47.235003 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:23:48.171387 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:48.171354 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-j988q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 07 13:23:48.171832 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:48.171408 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" podUID="c6c35371-5324-4554-9831-222b6cde861a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 07 13:23:49.171345 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:49.171311 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-j988q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 07 13:23:49.171520 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:49.171371 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" podUID="c6c35371-5324-4554-9831-222b6cde861a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 07 13:23:49.234166 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:49.234132 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:23:50.171555 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:50.171521 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-j988q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 07 13:23:50.171974 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:50.171602 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" podUID="c6c35371-5324-4554-9831-222b6cde861a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 07 13:23:50.221932 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:50.221906 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:23:51.175182 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:51.175145 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:23:51.176244 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:51.176225 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-j988q" Jul 07 13:23:51.687636 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:51.687603 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-llz2n" Jul 07 13:23:51.687832 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:51.687668 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-z95v9" Jul 07 13:23:55.101505 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.101473 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s"] Jul 07 13:23:55.104513 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.104494 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.108377 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.108361 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jn6x5\"" Jul 07 13:23:55.108481 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.108408 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 13:23:55.108481 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.108445 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 13:23:55.108636 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.108481 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 13:23:55.117683 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.117660 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s"] Jul 07 13:23:55.151172 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.151146 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/212d7df7-0e6e-4993-a436-c84bdf87ea94-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.151339 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.151183 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/212d7df7-0e6e-4993-a436-c84bdf87ea94-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.151339 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.151210 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/212d7df7-0e6e-4993-a436-c84bdf87ea94-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.151339 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.151235 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/212d7df7-0e6e-4993-a436-c84bdf87ea94-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.151339 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.151304 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcsxc\" (UniqueName: \"kubernetes.io/projected/212d7df7-0e6e-4993-a436-c84bdf87ea94-kube-api-access-pcsxc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.151517 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.151362 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/212d7df7-0e6e-4993-a436-c84bdf87ea94-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.178637 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.178571 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:23:55.252713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.252678 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/212d7df7-0e6e-4993-a436-c84bdf87ea94-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.252713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.252719 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/212d7df7-0e6e-4993-a436-c84bdf87ea94-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.252917 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.252744 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/212d7df7-0e6e-4993-a436-c84bdf87ea94-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.252917 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.252763 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/212d7df7-0e6e-4993-a436-c84bdf87ea94-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.252917 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.252873 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcsxc\" (UniqueName: \"kubernetes.io/projected/212d7df7-0e6e-4993-a436-c84bdf87ea94-kube-api-access-pcsxc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.253060 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.252937 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/212d7df7-0e6e-4993-a436-c84bdf87ea94-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.253127 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.253096 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/212d7df7-0e6e-4993-a436-c84bdf87ea94-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.253241 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.253144 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/212d7df7-0e6e-4993-a436-c84bdf87ea94-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.253431 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.253408 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/212d7df7-0e6e-4993-a436-c84bdf87ea94-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.255094 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.255076 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/212d7df7-0e6e-4993-a436-c84bdf87ea94-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.255434 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.255416 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/212d7df7-0e6e-4993-a436-c84bdf87ea94-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.262662 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.262637 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcsxc\" (UniqueName: \"kubernetes.io/projected/212d7df7-0e6e-4993-a436-c84bdf87ea94-kube-api-access-pcsxc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s\" (UID: \"212d7df7-0e6e-4993-a436-c84bdf87ea94\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.414961 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.414929 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:23:55.544469 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.544439 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s"] Jul 07 13:23:55.546145 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:23:55.546110 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod212d7df7_0e6e_4993_a436_c84bdf87ea94.slice/crio-9e9e744717b4c44117f8139ef2612309636f8fe9aac442114ddfe8e6c4f82171 WatchSource:0}: Error finding container 9e9e744717b4c44117f8139ef2612309636f8fe9aac442114ddfe8e6c4f82171: Status 404 returned error can't find the container with id 9e9e744717b4c44117f8139ef2612309636f8fe9aac442114ddfe8e6c4f82171 Jul 07 13:23:55.772680 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:55.772601 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" event={"ID":"212d7df7-0e6e-4993-a436-c84bdf87ea94","Type":"ContainerStarted","Data":"9e9e744717b4c44117f8139ef2612309636f8fe9aac442114ddfe8e6c4f82171"} Jul 07 13:23:56.779287 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:56.779253 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" event={"ID":"212d7df7-0e6e-4993-a436-c84bdf87ea94","Type":"ContainerStarted","Data":"8a03e4be531656b80d3e388ab19cc66d446ef4ecfd520733872e74467763181c"} Jul 07 13:23:56.809023 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:23:56.808973 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" podStartSLOduration=0.649146692 podStartE2EDuration="1.808955966s" totalImagesPullingTime="1.159809274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:23:56.807973205 +0000 UTC m=+594.486534540" watchObservedRunningTime="2026-07-07 13:23:56.808955966 +0000 UTC m=+594.487517299" Jul 07 13:24:01.349238 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:01.349199 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:24:05.415096 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:05.415059 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:24:05.415530 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:05.415123 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:24:05.427242 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:05.427208 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:24:05.839193 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:05.839097 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s" Jul 07 13:24:12.890654 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:12.890620 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:24:13.792070 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:13.792037 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:24:28.508397 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.508326 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw"] Jul 07 13:24:28.511277 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.511262 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.515563 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.515542 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 07 13:24:28.524451 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.524423 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw"] Jul 07 13:24:28.553218 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.553194 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:24:28.580063 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.580020 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d3f206c-2eac-4918-9d73-5b83642e56be-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.580199 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.580103 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw8qr\" (UniqueName: \"kubernetes.io/projected/8d3f206c-2eac-4918-9d73-5b83642e56be-kube-api-access-rw8qr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.580258 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.580208 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d3f206c-2eac-4918-9d73-5b83642e56be-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.580309 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.580260 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d3f206c-2eac-4918-9d73-5b83642e56be-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.580395 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.580361 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d3f206c-2eac-4918-9d73-5b83642e56be-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.580445 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.580435 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d3f206c-2eac-4918-9d73-5b83642e56be-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.681491 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.681461 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d3f206c-2eac-4918-9d73-5b83642e56be-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.681698 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.681505 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d3f206c-2eac-4918-9d73-5b83642e56be-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.681698 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.681529 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw8qr\" (UniqueName: \"kubernetes.io/projected/8d3f206c-2eac-4918-9d73-5b83642e56be-kube-api-access-rw8qr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.681698 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.681603 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d3f206c-2eac-4918-9d73-5b83642e56be-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.681698 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.681629 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d3f206c-2eac-4918-9d73-5b83642e56be-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.681698 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.681691 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d3f206c-2eac-4918-9d73-5b83642e56be-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.682061 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.682033 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d3f206c-2eac-4918-9d73-5b83642e56be-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.682157 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.682061 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d3f206c-2eac-4918-9d73-5b83642e56be-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.682157 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.682070 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d3f206c-2eac-4918-9d73-5b83642e56be-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.683714 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.683690 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d3f206c-2eac-4918-9d73-5b83642e56be-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.684882 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.684862 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d3f206c-2eac-4918-9d73-5b83642e56be-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.697332 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.697309 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw8qr\" (UniqueName: \"kubernetes.io/projected/8d3f206c-2eac-4918-9d73-5b83642e56be-kube-api-access-rw8qr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw\" (UID: \"8d3f206c-2eac-4918-9d73-5b83642e56be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.821741 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.821661 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:28.956158 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.955986 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw"] Jul 07 13:24:28.959092 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:24:28.959063 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d3f206c_2eac_4918_9d73_5b83642e56be.slice/crio-8aa238aae76fd2fef8c44a443a4236f3f4d71ac6624f9d203a945bb3b5efaa80 WatchSource:0}: Error finding container 8aa238aae76fd2fef8c44a443a4236f3f4d71ac6624f9d203a945bb3b5efaa80: Status 404 returned error can't find the container with id 8aa238aae76fd2fef8c44a443a4236f3f4d71ac6624f9d203a945bb3b5efaa80 Jul 07 13:24:28.960358 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:28.960335 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:24:29.955069 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:29.955029 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" event={"ID":"8d3f206c-2eac-4918-9d73-5b83642e56be","Type":"ContainerStarted","Data":"bddff45408a36336596c8ab16ec9291b3666202ff3f207b23db6af1bbb26564e"} Jul 07 13:24:29.955437 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:29.955089 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" event={"ID":"8d3f206c-2eac-4918-9d73-5b83642e56be","Type":"ContainerStarted","Data":"8aa238aae76fd2fef8c44a443a4236f3f4d71ac6624f9d203a945bb3b5efaa80"} Jul 07 13:24:29.989025 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:29.988981 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" podStartSLOduration=1.767598953 podStartE2EDuration="1.988964675s" totalImagesPullingTime="221.365722ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:24:29.983705453 +0000 UTC m=+627.662266785" watchObservedRunningTime="2026-07-07 13:24:29.988964675 +0000 UTC m=+627.667526006" Jul 07 13:24:37.398465 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.398421 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:24:37.504018 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.503983 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:24:37.512989 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.512963 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf"] Jul 07 13:24:37.517333 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.517313 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.525167 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.525145 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 07 13:24:37.533073 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.533049 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf"] Jul 07 13:24:37.568899 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.568864 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlsdb\" (UniqueName: \"kubernetes.io/projected/d0aa835b-3bc1-45ad-971b-5d89199d7d91-kube-api-access-vlsdb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.569042 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.568929 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0aa835b-3bc1-45ad-971b-5d89199d7d91-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.569042 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.568981 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0aa835b-3bc1-45ad-971b-5d89199d7d91-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.569042 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.569030 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0aa835b-3bc1-45ad-971b-5d89199d7d91-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.569149 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.569128 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0aa835b-3bc1-45ad-971b-5d89199d7d91-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.569212 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.569194 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0aa835b-3bc1-45ad-971b-5d89199d7d91-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.629476 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.629442 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:24:37.670629 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.670597 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0aa835b-3bc1-45ad-971b-5d89199d7d91-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.670787 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.670686 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlsdb\" (UniqueName: \"kubernetes.io/projected/d0aa835b-3bc1-45ad-971b-5d89199d7d91-kube-api-access-vlsdb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.670787 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.670712 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0aa835b-3bc1-45ad-971b-5d89199d7d91-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.670787 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.670757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0aa835b-3bc1-45ad-971b-5d89199d7d91-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.670939 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.670794 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0aa835b-3bc1-45ad-971b-5d89199d7d91-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.670939 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.670839 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0aa835b-3bc1-45ad-971b-5d89199d7d91-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.671196 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.671174 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0aa835b-3bc1-45ad-971b-5d89199d7d91-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.671297 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.671225 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0aa835b-3bc1-45ad-971b-5d89199d7d91-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.671297 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.671247 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0aa835b-3bc1-45ad-971b-5d89199d7d91-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.672939 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.672917 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0aa835b-3bc1-45ad-971b-5d89199d7d91-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.673148 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.673128 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0aa835b-3bc1-45ad-971b-5d89199d7d91-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.678685 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.678661 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlsdb\" (UniqueName: \"kubernetes.io/projected/d0aa835b-3bc1-45ad-971b-5d89199d7d91-kube-api-access-vlsdb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s5jhf\" (UID: \"d0aa835b-3bc1-45ad-971b-5d89199d7d91\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.828550 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.828509 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:37.968353 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.968277 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf"] Jul 07 13:24:37.971755 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:24:37.971726 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0aa835b_3bc1_45ad_971b_5d89199d7d91.slice/crio-cd7dfae2c88d23ef6c68a7d5fef3719da5c06215fd024a38fbf243b592cff144 WatchSource:0}: Error finding container cd7dfae2c88d23ef6c68a7d5fef3719da5c06215fd024a38fbf243b592cff144: Status 404 returned error can't find the container with id cd7dfae2c88d23ef6c68a7d5fef3719da5c06215fd024a38fbf243b592cff144 Jul 07 13:24:37.996613 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:37.996555 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" event={"ID":"d0aa835b-3bc1-45ad-971b-5d89199d7d91","Type":"ContainerStarted","Data":"cd7dfae2c88d23ef6c68a7d5fef3719da5c06215fd024a38fbf243b592cff144"} Jul 07 13:24:38.822682 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:38.822646 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:38.823059 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:38.822730 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:38.835065 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:38.835034 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:39.002845 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:39.002807 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" event={"ID":"d0aa835b-3bc1-45ad-971b-5d89199d7d91","Type":"ContainerStarted","Data":"d06e1daa371215712e471df3b8dceffbe9ff2e7a076c1857ed37409436a1431b"} Jul 07 13:24:39.014162 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:39.014132 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw" Jul 07 13:24:39.025770 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:39.025719 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" podStartSLOduration=1.758746227 podStartE2EDuration="2.025704357s" totalImagesPullingTime="266.95813ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:24:39.023996416 +0000 UTC m=+636.702557749" watchObservedRunningTime="2026-07-07 13:24:39.025704357 +0000 UTC m=+636.704265688" Jul 07 13:24:44.515591 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:44.515543 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:24:47.829525 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:47.829491 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:47.829918 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:47.829548 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:47.844342 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:47.844316 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:24:48.058118 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:24:48.058091 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s5jhf" Jul 07 13:25:51.694600 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:25:51.694539 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:25:51.795979 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:25:51.795943 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:25:53.026054 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:25:53.026028 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:25:53.129235 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:25:53.129194 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:26:48.271201 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:26:48.271156 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:03.732985 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:03.732950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:03.797816 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:03.797782 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:04.942249 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:04.942211 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:05.061848 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.061814 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-777d69b966-mp8xq"] Jul 07 13:27:05.062059 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.062032 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-777d69b966-mp8xq" podUID="8bf6965e-2cc6-42b0-8e16-45c8838cfab6" containerName="manager" containerID="cri-o://26eac01b1e64faae5ff5b4e77d17c6431216e23a10d9ba38933ca1adb293d6a9" gracePeriod=30 Jul 07 13:27:05.309285 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.309262 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-777d69b966-mp8xq" Jul 07 13:27:05.457451 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.457415 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf6965e-2cc6-42b0-8e16-45c8838cfab6-webhook-cert\") pod \"8bf6965e-2cc6-42b0-8e16-45c8838cfab6\" (UID: \"8bf6965e-2cc6-42b0-8e16-45c8838cfab6\") " Jul 07 13:27:05.457668 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.457502 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtzzx\" (UniqueName: \"kubernetes.io/projected/8bf6965e-2cc6-42b0-8e16-45c8838cfab6-kube-api-access-mtzzx\") pod \"8bf6965e-2cc6-42b0-8e16-45c8838cfab6\" (UID: \"8bf6965e-2cc6-42b0-8e16-45c8838cfab6\") " Jul 07 13:27:05.459475 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.459435 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bf6965e-2cc6-42b0-8e16-45c8838cfab6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8bf6965e-2cc6-42b0-8e16-45c8838cfab6" (UID: "8bf6965e-2cc6-42b0-8e16-45c8838cfab6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:27:05.459626 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.459503 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bf6965e-2cc6-42b0-8e16-45c8838cfab6-kube-api-access-mtzzx" (OuterVolumeSpecName: "kube-api-access-mtzzx") pod "8bf6965e-2cc6-42b0-8e16-45c8838cfab6" (UID: "8bf6965e-2cc6-42b0-8e16-45c8838cfab6"). InnerVolumeSpecName "kube-api-access-mtzzx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:27:05.559016 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.558928 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtzzx\" (UniqueName: \"kubernetes.io/projected/8bf6965e-2cc6-42b0-8e16-45c8838cfab6-kube-api-access-mtzzx\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:27:05.559016 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.558956 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf6965e-2cc6-42b0-8e16-45c8838cfab6-webhook-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:27:05.747997 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.747960 2568 generic.go:356] "Generic (PLEG): container finished" podID="8bf6965e-2cc6-42b0-8e16-45c8838cfab6" containerID="26eac01b1e64faae5ff5b4e77d17c6431216e23a10d9ba38933ca1adb293d6a9" exitCode=0 Jul 07 13:27:05.748160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.748020 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-777d69b966-mp8xq" Jul 07 13:27:05.748160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.748043 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-777d69b966-mp8xq" event={"ID":"8bf6965e-2cc6-42b0-8e16-45c8838cfab6","Type":"ContainerDied","Data":"26eac01b1e64faae5ff5b4e77d17c6431216e23a10d9ba38933ca1adb293d6a9"} Jul 07 13:27:05.748160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.748119 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-777d69b966-mp8xq" event={"ID":"8bf6965e-2cc6-42b0-8e16-45c8838cfab6","Type":"ContainerDied","Data":"187b833943667c14d21801353b2ea62c1ff0d85ffd268b1139c3637f80e53ef9"} Jul 07 13:27:05.748160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.748142 2568 scope.go:117] "RemoveContainer" containerID="26eac01b1e64faae5ff5b4e77d17c6431216e23a10d9ba38933ca1adb293d6a9" Jul 07 13:27:05.757909 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.757892 2568 scope.go:117] "RemoveContainer" containerID="26eac01b1e64faae5ff5b4e77d17c6431216e23a10d9ba38933ca1adb293d6a9" Jul 07 13:27:05.758149 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:27:05.758133 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26eac01b1e64faae5ff5b4e77d17c6431216e23a10d9ba38933ca1adb293d6a9\": container with ID starting with 26eac01b1e64faae5ff5b4e77d17c6431216e23a10d9ba38933ca1adb293d6a9 not found: ID does not exist" containerID="26eac01b1e64faae5ff5b4e77d17c6431216e23a10d9ba38933ca1adb293d6a9" Jul 07 13:27:05.758187 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.758158 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26eac01b1e64faae5ff5b4e77d17c6431216e23a10d9ba38933ca1adb293d6a9"} err="failed to get container status \"26eac01b1e64faae5ff5b4e77d17c6431216e23a10d9ba38933ca1adb293d6a9\": rpc error: code = NotFound desc = could not find container \"26eac01b1e64faae5ff5b4e77d17c6431216e23a10d9ba38933ca1adb293d6a9\": container with ID starting with 26eac01b1e64faae5ff5b4e77d17c6431216e23a10d9ba38933ca1adb293d6a9 not found: ID does not exist" Jul 07 13:27:05.770818 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.770796 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-777d69b966-mp8xq"] Jul 07 13:27:05.777488 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:05.777469 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-777d69b966-mp8xq"] Jul 07 13:27:06.517811 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.517777 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:06.637789 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.637758 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-777d69b966-pshl6"] Jul 07 13:27:06.638236 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.638223 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bf6965e-2cc6-42b0-8e16-45c8838cfab6" containerName="manager" Jul 07 13:27:06.638236 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.638237 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf6965e-2cc6-42b0-8e16-45c8838cfab6" containerName="manager" Jul 07 13:27:06.638336 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.638306 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bf6965e-2cc6-42b0-8e16-45c8838cfab6" containerName="manager" Jul 07 13:27:06.641237 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.641219 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-777d69b966-pshl6" Jul 07 13:27:06.644172 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.644150 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 13:27:06.645828 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.645805 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 13:27:06.645973 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.645949 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pzvhp\"" Jul 07 13:27:06.653241 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.653215 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-777d69b966-pshl6"] Jul 07 13:27:06.772296 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.772207 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2f49512f-fc30-4a66-819d-7ba4a2e0a1e8-webhook-cert\") pod \"maas-controller-777d69b966-pshl6\" (UID: \"2f49512f-fc30-4a66-819d-7ba4a2e0a1e8\") " pod="opendatahub/maas-controller-777d69b966-pshl6" Jul 07 13:27:06.772296 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.772267 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pwzm\" (UniqueName: \"kubernetes.io/projected/2f49512f-fc30-4a66-819d-7ba4a2e0a1e8-kube-api-access-7pwzm\") pod \"maas-controller-777d69b966-pshl6\" (UID: \"2f49512f-fc30-4a66-819d-7ba4a2e0a1e8\") " pod="opendatahub/maas-controller-777d69b966-pshl6" Jul 07 13:27:06.873791 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.873763 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pwzm\" (UniqueName: \"kubernetes.io/projected/2f49512f-fc30-4a66-819d-7ba4a2e0a1e8-kube-api-access-7pwzm\") pod \"maas-controller-777d69b966-pshl6\" (UID: \"2f49512f-fc30-4a66-819d-7ba4a2e0a1e8\") " pod="opendatahub/maas-controller-777d69b966-pshl6" Jul 07 13:27:06.873956 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.873865 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2f49512f-fc30-4a66-819d-7ba4a2e0a1e8-webhook-cert\") pod \"maas-controller-777d69b966-pshl6\" (UID: \"2f49512f-fc30-4a66-819d-7ba4a2e0a1e8\") " pod="opendatahub/maas-controller-777d69b966-pshl6" Jul 07 13:27:06.876189 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.876163 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2f49512f-fc30-4a66-819d-7ba4a2e0a1e8-webhook-cert\") pod \"maas-controller-777d69b966-pshl6\" (UID: \"2f49512f-fc30-4a66-819d-7ba4a2e0a1e8\") " pod="opendatahub/maas-controller-777d69b966-pshl6" Jul 07 13:27:06.883322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.883285 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pwzm\" (UniqueName: \"kubernetes.io/projected/2f49512f-fc30-4a66-819d-7ba4a2e0a1e8-kube-api-access-7pwzm\") pod \"maas-controller-777d69b966-pshl6\" (UID: \"2f49512f-fc30-4a66-819d-7ba4a2e0a1e8\") " pod="opendatahub/maas-controller-777d69b966-pshl6" Jul 07 13:27:06.954913 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.954878 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-777d69b966-pshl6" Jul 07 13:27:06.972654 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:06.972623 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bf6965e-2cc6-42b0-8e16-45c8838cfab6" path="/var/lib/kubelet/pods/8bf6965e-2cc6-42b0-8e16-45c8838cfab6/volumes" Jul 07 13:27:07.089061 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:07.089040 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-777d69b966-pshl6"] Jul 07 13:27:07.090737 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:27:07.090708 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f49512f_fc30_4a66_819d_7ba4a2e0a1e8.slice/crio-d50101f08e6255ff0dcc626063011fff1a2fea33a74b79746b52e159e8d5963c WatchSource:0}: Error finding container d50101f08e6255ff0dcc626063011fff1a2fea33a74b79746b52e159e8d5963c: Status 404 returned error can't find the container with id d50101f08e6255ff0dcc626063011fff1a2fea33a74b79746b52e159e8d5963c Jul 07 13:27:07.761624 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:07.761518 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-777d69b966-pshl6" event={"ID":"2f49512f-fc30-4a66-819d-7ba4a2e0a1e8","Type":"ContainerStarted","Data":"9b57001df4e8e4b859873ac7993d0e8dfaa525bc8a6238627f1d2782857f844b"} Jul 07 13:27:07.761624 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:07.761570 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-777d69b966-pshl6" event={"ID":"2f49512f-fc30-4a66-819d-7ba4a2e0a1e8","Type":"ContainerStarted","Data":"d50101f08e6255ff0dcc626063011fff1a2fea33a74b79746b52e159e8d5963c"} Jul 07 13:27:07.761624 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:07.761618 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-777d69b966-pshl6" Jul 07 13:27:07.781833 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:07.781789 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-777d69b966-pshl6" podStartSLOduration=1.380677085 podStartE2EDuration="1.781774146s" totalImagesPullingTime="401.097061ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:27:07.779901915 +0000 UTC m=+785.458463245" watchObservedRunningTime="2026-07-07 13:27:07.781774146 +0000 UTC m=+785.460335477" Jul 07 13:27:18.773406 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:18.773371 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-777d69b966-pshl6" Jul 07 13:27:20.389700 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:20.389666 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:20.508437 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:20.508406 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:37.727641 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:37.727536 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:37.826003 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:37.825972 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:45.697830 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:45.697796 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:45.796746 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:45.796711 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:45.868893 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:45.868863 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:51.940250 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:51.940213 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:52.082325 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:52.082293 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:52.177330 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:52.177298 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:52.295701 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:52.295621 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:58.087237 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:58.087184 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:27:58.153459 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:27:58.153405 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:30:17.943123 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:30:17.943093 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:31:34.851029 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:31:34.850990 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:33:10.391259 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:33:10.391229 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:37:31.130670 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:37:31.130630 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:37:32.462851 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:37:32.462816 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:38:21.868921 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:38:21.868879 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:38:46.524187 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:38:46.524142 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:38:55.380761 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:38:55.380722 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:03.666007 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:03.665915 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:08.837809 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:08.837772 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:11.844647 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:11.844613 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:11.951901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:11.951864 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:12.315586 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:12.315545 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:18.202319 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.202283 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:18.305898 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.305860 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:18.422894 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.422867 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9"] Jul 07 13:39:18.426623 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.426556 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.434619 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.434597 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-31f94700-openshift-default-dockercfg-sf7z9\"" Jul 07 13:39:18.459791 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.459721 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9"] Jul 07 13:39:18.470531 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.470492 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:18.543009 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.542972 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-podinfo\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.543188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.543017 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-workload-socket\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.543188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.543082 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-workload-certs\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.543188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.543104 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istiod-ca-cert\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.543188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.543176 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-credential-socket\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.543342 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.543207 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-token\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.543342 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.543298 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-envoy\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.543342 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.543327 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-data\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.543456 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.543353 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbjfd\" (UniqueName: \"kubernetes.io/projected/b11cf8f1-be47-4486-a07d-a1c0825a4b44-kube-api-access-xbjfd\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.572708 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.572681 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:18.644185 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.644148 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-data\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.644358 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.644195 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbjfd\" (UniqueName: \"kubernetes.io/projected/b11cf8f1-be47-4486-a07d-a1c0825a4b44-kube-api-access-xbjfd\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.644358 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.644320 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-podinfo\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.644358 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.644352 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-workload-socket\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.644519 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.644387 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-workload-certs\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.644519 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.644498 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istiod-ca-cert\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.644672 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.644588 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-credential-socket\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.644672 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.644629 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-token\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.644672 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.644634 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-data\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.644672 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.644662 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-workload-certs\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.644893 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.644741 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-workload-socket\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.644893 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.644753 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-envoy\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.644973 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.644889 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-credential-socket\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.645204 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.645180 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istiod-ca-cert\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.646717 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.646692 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-podinfo\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.646830 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.646777 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-envoy\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.653218 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.653194 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-token\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.653461 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.653445 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbjfd\" (UniqueName: \"kubernetes.io/projected/b11cf8f1-be47-4486-a07d-a1c0825a4b44-kube-api-access-xbjfd\") pod \"e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.737832 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.737757 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:18.884589 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.884548 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9"] Jul 07 13:39:18.886501 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:39:18.886473 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb11cf8f1_be47_4486_a07d_a1c0825a4b44.slice/crio-130fa60528234b5372ddcae1f424941b00dcc8e037dc73749310e4e9db4e5de4 WatchSource:0}: Error finding container 130fa60528234b5372ddcae1f424941b00dcc8e037dc73749310e4e9db4e5de4: Status 404 returned error can't find the container with id 130fa60528234b5372ddcae1f424941b00dcc8e037dc73749310e4e9db4e5de4 Jul 07 13:39:18.888665 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.888627 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:39:18.888788 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.888712 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:39:18.888788 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:18.888755 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:39:19.537851 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:19.537810 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" event={"ID":"b11cf8f1-be47-4486-a07d-a1c0825a4b44","Type":"ContainerStarted","Data":"05d2a734b049d640ea22238dc084bc37e121d5fb260358dade6708dc8c5a9b2b"} Jul 07 13:39:19.538339 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:19.537881 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" event={"ID":"b11cf8f1-be47-4486-a07d-a1c0825a4b44","Type":"ContainerStarted","Data":"130fa60528234b5372ddcae1f424941b00dcc8e037dc73749310e4e9db4e5de4"} Jul 07 13:39:19.579507 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:19.579472 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:19.690819 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:19.690782 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:19.738467 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:19.738436 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:19.743333 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:19.743306 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:19.770055 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:19.770000 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" podStartSLOduration=1.769980543 podStartE2EDuration="1.769980543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:39:19.575755381 +0000 UTC m=+1517.254316747" watchObservedRunningTime="2026-07-07 13:39:19.769980543 +0000 UTC m=+1517.448541873" Jul 07 13:39:19.801717 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:19.801637 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:20.260247 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:20.260213 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:20.371850 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:20.371814 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:20.544001 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:20.543912 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:20.545023 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:20.545004 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:21.948820 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:21.948791 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:24.727283 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:24.727212 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:24.906544 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:24.906509 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9"] Jul 07 13:39:24.906740 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:24.906714 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" podUID="b11cf8f1-be47-4486-a07d-a1c0825a4b44" containerName="istio-proxy" containerID="cri-o://05d2a734b049d640ea22238dc084bc37e121d5fb260358dade6708dc8c5a9b2b" gracePeriod=30 Jul 07 13:39:24.935887 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:24.935837 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:25.036117 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:25.036012 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:25.098787 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:25.098758 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:26.887598 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:26.887544 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:30.163327 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.163302 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:30.277778 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.277683 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-workload-certs\") pod \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " Jul 07 13:39:30.277778 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.277754 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-data\") pod \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " Jul 07 13:39:30.278002 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.277786 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-credential-socket\") pod \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " Jul 07 13:39:30.278002 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.277835 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-podinfo\") pod \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " Jul 07 13:39:30.278002 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.277864 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-envoy\") pod \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " Jul 07 13:39:30.278002 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.277894 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbjfd\" (UniqueName: \"kubernetes.io/projected/b11cf8f1-be47-4486-a07d-a1c0825a4b44-kube-api-access-xbjfd\") pod \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " Jul 07 13:39:30.278002 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.277915 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b11cf8f1-be47-4486-a07d-a1c0825a4b44" (UID: "b11cf8f1-be47-4486-a07d-a1c0825a4b44"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:39:30.278002 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.277924 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-workload-socket\") pod \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " Jul 07 13:39:30.278301 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.278025 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istiod-ca-cert\") pod \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " Jul 07 13:39:30.278301 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.278030 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-data" (OuterVolumeSpecName: "istio-data") pod "b11cf8f1-be47-4486-a07d-a1c0825a4b44" (UID: "b11cf8f1-be47-4486-a07d-a1c0825a4b44"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:39:30.278301 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.278054 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-token\") pod \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\" (UID: \"b11cf8f1-be47-4486-a07d-a1c0825a4b44\") " Jul 07 13:39:30.278301 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.278071 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b11cf8f1-be47-4486-a07d-a1c0825a4b44" (UID: "b11cf8f1-be47-4486-a07d-a1c0825a4b44"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:39:30.278301 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.278183 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b11cf8f1-be47-4486-a07d-a1c0825a4b44" (UID: "b11cf8f1-be47-4486-a07d-a1c0825a4b44"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:39:30.278525 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.278358 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b11cf8f1-be47-4486-a07d-a1c0825a4b44" (UID: "b11cf8f1-be47-4486-a07d-a1c0825a4b44"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:39:30.278525 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.278499 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-workload-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:30.278525 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.278517 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istiod-ca-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:30.278691 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.278530 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-workload-certs\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:30.278691 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.278542 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:30.278691 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.278556 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-credential-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:30.280220 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.280191 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b11cf8f1-be47-4486-a07d-a1c0825a4b44" (UID: "b11cf8f1-be47-4486-a07d-a1c0825a4b44"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:39:30.280329 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.280221 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-token" (OuterVolumeSpecName: "istio-token") pod "b11cf8f1-be47-4486-a07d-a1c0825a4b44" (UID: "b11cf8f1-be47-4486-a07d-a1c0825a4b44"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:39:30.280329 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.280313 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b11cf8f1-be47-4486-a07d-a1c0825a4b44" (UID: "b11cf8f1-be47-4486-a07d-a1c0825a4b44"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:39:30.280401 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.280323 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b11cf8f1-be47-4486-a07d-a1c0825a4b44-kube-api-access-xbjfd" (OuterVolumeSpecName: "kube-api-access-xbjfd") pod "b11cf8f1-be47-4486-a07d-a1c0825a4b44" (UID: "b11cf8f1-be47-4486-a07d-a1c0825a4b44"). InnerVolumeSpecName "kube-api-access-xbjfd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:39:30.379062 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.379032 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-podinfo\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:30.379062 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.379057 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-envoy\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:30.379062 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.379066 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbjfd\" (UniqueName: \"kubernetes.io/projected/b11cf8f1-be47-4486-a07d-a1c0825a4b44-kube-api-access-xbjfd\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:30.379261 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.379074 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b11cf8f1-be47-4486-a07d-a1c0825a4b44-istio-token\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:30.595552 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.595457 2568 generic.go:356] "Generic (PLEG): container finished" podID="b11cf8f1-be47-4486-a07d-a1c0825a4b44" containerID="05d2a734b049d640ea22238dc084bc37e121d5fb260358dade6708dc8c5a9b2b" exitCode=0 Jul 07 13:39:30.595552 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.595507 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" Jul 07 13:39:30.595815 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.595547 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" event={"ID":"b11cf8f1-be47-4486-a07d-a1c0825a4b44","Type":"ContainerDied","Data":"05d2a734b049d640ea22238dc084bc37e121d5fb260358dade6708dc8c5a9b2b"} Jul 07 13:39:30.595815 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.595637 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9" event={"ID":"b11cf8f1-be47-4486-a07d-a1c0825a4b44","Type":"ContainerDied","Data":"130fa60528234b5372ddcae1f424941b00dcc8e037dc73749310e4e9db4e5de4"} Jul 07 13:39:30.595815 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.595669 2568 scope.go:117] "RemoveContainer" containerID="05d2a734b049d640ea22238dc084bc37e121d5fb260358dade6708dc8c5a9b2b" Jul 07 13:39:30.605177 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.605161 2568 scope.go:117] "RemoveContainer" containerID="05d2a734b049d640ea22238dc084bc37e121d5fb260358dade6708dc8c5a9b2b" Jul 07 13:39:30.605429 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:39:30.605409 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05d2a734b049d640ea22238dc084bc37e121d5fb260358dade6708dc8c5a9b2b\": container with ID starting with 05d2a734b049d640ea22238dc084bc37e121d5fb260358dade6708dc8c5a9b2b not found: ID does not exist" containerID="05d2a734b049d640ea22238dc084bc37e121d5fb260358dade6708dc8c5a9b2b" Jul 07 13:39:30.605476 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.605439 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05d2a734b049d640ea22238dc084bc37e121d5fb260358dade6708dc8c5a9b2b"} err="failed to get container status \"05d2a734b049d640ea22238dc084bc37e121d5fb260358dade6708dc8c5a9b2b\": rpc error: code = NotFound desc = could not find container \"05d2a734b049d640ea22238dc084bc37e121d5fb260358dade6708dc8c5a9b2b\": container with ID starting with 05d2a734b049d640ea22238dc084bc37e121d5fb260358dade6708dc8c5a9b2b not found: ID does not exist" Jul 07 13:39:30.618726 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.618705 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9"] Jul 07 13:39:30.621699 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.621680 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-31f94700-openshift-default-ff7995f87-dpzv9"] Jul 07 13:39:30.972064 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:30.972032 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b11cf8f1-be47-4486-a07d-a1c0825a4b44" path="/var/lib/kubelet/pods/b11cf8f1-be47-4486-a07d-a1c0825a4b44/volumes" Jul 07 13:39:31.080708 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.080671 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:31.186282 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.186248 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:31.268435 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.268359 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg"] Jul 07 13:39:31.268883 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.268867 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b11cf8f1-be47-4486-a07d-a1c0825a4b44" containerName="istio-proxy" Jul 07 13:39:31.268883 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.268883 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b11cf8f1-be47-4486-a07d-a1c0825a4b44" containerName="istio-proxy" Jul 07 13:39:31.269049 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.268964 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b11cf8f1-be47-4486-a07d-a1c0825a4b44" containerName="istio-proxy" Jul 07 13:39:31.274749 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.274728 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.277341 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.277322 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-de419fb7-openshift-default-dockercfg-vhtv8\"" Jul 07 13:39:31.289923 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.289894 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg"] Jul 07 13:39:31.319619 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.319565 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:31.389824 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.389782 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc7bdc96-fc50-4396-936a-f41cee5f3557-istiod-ca-cert\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.390026 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.389868 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-workload-socket\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.390026 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.389907 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-workload-certs\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.390026 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.389930 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-token\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.390026 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.389951 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-data\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.390179 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.390061 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-podinfo\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.390179 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.390120 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksh2r\" (UniqueName: \"kubernetes.io/projected/dc7bdc96-fc50-4396-936a-f41cee5f3557-kube-api-access-ksh2r\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.390179 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.390150 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-envoy\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.390285 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.390184 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-credential-socket\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.427639 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.427610 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:31.491447 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.491420 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksh2r\" (UniqueName: \"kubernetes.io/projected/dc7bdc96-fc50-4396-936a-f41cee5f3557-kube-api-access-ksh2r\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.491622 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.491459 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-envoy\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.491622 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.491490 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-credential-socket\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.491622 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.491513 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc7bdc96-fc50-4396-936a-f41cee5f3557-istiod-ca-cert\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.491622 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.491549 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-workload-socket\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.491622 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.491568 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-workload-certs\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.491896 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.491664 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-token\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.491896 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.491703 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-data\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.491896 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.491822 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-podinfo\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.492045 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.491956 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-workload-socket\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.492045 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.492014 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-credential-socket\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.492144 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.492042 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-data\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.492144 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.492080 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-workload-certs\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.492723 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.492697 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc7bdc96-fc50-4396-936a-f41cee5f3557-istiod-ca-cert\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.493824 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.493807 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-envoy\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.494246 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.494230 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-podinfo\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.513945 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.513920 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-token\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.514107 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.514089 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksh2r\" (UniqueName: \"kubernetes.io/projected/dc7bdc96-fc50-4396-936a-f41cee5f3557-kube-api-access-ksh2r\") pod \"e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.588019 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.587940 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:31.741531 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.741501 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg"] Jul 07 13:39:31.742636 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:39:31.742614 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc7bdc96_fc50_4396_936a_f41cee5f3557.slice/crio-71975019c2ecda287c0d97f39f7adb168b242732041dfdb3b8e6d2672013258a WatchSource:0}: Error finding container 71975019c2ecda287c0d97f39f7adb168b242732041dfdb3b8e6d2672013258a: Status 404 returned error can't find the container with id 71975019c2ecda287c0d97f39f7adb168b242732041dfdb3b8e6d2672013258a Jul 07 13:39:31.744830 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.744797 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:39:31.744902 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.744863 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:39:31.744902 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:31.744889 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:39:32.404845 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:32.404811 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:32.489936 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:32.489901 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:32.563478 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:32.563446 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:32.609037 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:32.609008 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" event={"ID":"dc7bdc96-fc50-4396-936a-f41cee5f3557","Type":"ContainerStarted","Data":"fd323009582e5b6ef951e9314cd8170132320134cbe1312d23dc769623b5b8ee"} Jul 07 13:39:32.609220 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:32.609054 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" event={"ID":"dc7bdc96-fc50-4396-936a-f41cee5f3557","Type":"ContainerStarted","Data":"71975019c2ecda287c0d97f39f7adb168b242732041dfdb3b8e6d2672013258a"} Jul 07 13:39:32.632458 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:32.632333 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" podStartSLOduration=1.632314623 podStartE2EDuration="1.632314623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:39:32.631185183 +0000 UTC m=+1530.309746514" watchObservedRunningTime="2026-07-07 13:39:32.632314623 +0000 UTC m=+1530.310875955" Jul 07 13:39:32.664473 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:32.664450 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:33.005337 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:33.005231 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:33.099905 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:33.099862 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:33.588591 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:33.588549 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:33.593855 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:33.593829 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:33.615105 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:33.615073 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:33.616040 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:33.616017 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:34.259720 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:34.259690 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:37.664693 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:37.664661 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:38.089808 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:38.089737 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:43.681467 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:43.681438 2568 generic.go:356] "Generic (PLEG): container finished" podID="dc7bdc96-fc50-4396-936a-f41cee5f3557" containerID="fd323009582e5b6ef951e9314cd8170132320134cbe1312d23dc769623b5b8ee" exitCode=0 Jul 07 13:39:43.681908 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:43.681511 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" event={"ID":"dc7bdc96-fc50-4396-936a-f41cee5f3557","Type":"ContainerDied","Data":"fd323009582e5b6ef951e9314cd8170132320134cbe1312d23dc769623b5b8ee"} Jul 07 13:39:43.682055 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:43.682038 2568 scope.go:117] "RemoveContainer" containerID="fd323009582e5b6ef951e9314cd8170132320134cbe1312d23dc769623b5b8ee" Jul 07 13:39:43.683079 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:43.683050 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:39:43.683166 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:43.683123 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:39:43.683166 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:43.683159 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:39:43.779361 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:43.779340 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg"] Jul 07 13:39:43.822664 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:43.822637 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:43.924759 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:43.924715 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:44.689134 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:44.689104 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" event={"ID":"dc7bdc96-fc50-4396-936a-f41cee5f3557","Type":"ContainerStarted","Data":"601bd14f5073a31fb7f03e012c60e577db78548e9bb86df72bd21240b4a6b7b6"} Jul 07 13:39:44.689553 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:44.689170 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" podUID="dc7bdc96-fc50-4396-936a-f41cee5f3557" containerName="istio-proxy" containerID="cri-o://601bd14f5073a31fb7f03e012c60e577db78548e9bb86df72bd21240b4a6b7b6" gracePeriod=30 Jul 07 13:39:48.615056 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:48.615026 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:49.917416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:49.917388 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:49.949518 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:49.949458 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:50.019933 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.019902 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:50.084644 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.084614 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-data\") pod \"dc7bdc96-fc50-4396-936a-f41cee5f3557\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " Jul 07 13:39:50.084828 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.084666 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc7bdc96-fc50-4396-936a-f41cee5f3557-istiod-ca-cert\") pod \"dc7bdc96-fc50-4396-936a-f41cee5f3557\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " Jul 07 13:39:50.084828 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.084697 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-workload-certs\") pod \"dc7bdc96-fc50-4396-936a-f41cee5f3557\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " Jul 07 13:39:50.084828 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.084762 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-credential-socket\") pod \"dc7bdc96-fc50-4396-936a-f41cee5f3557\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " Jul 07 13:39:50.084828 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.084799 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-envoy\") pod \"dc7bdc96-fc50-4396-936a-f41cee5f3557\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " Jul 07 13:39:50.085099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.084838 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-podinfo\") pod \"dc7bdc96-fc50-4396-936a-f41cee5f3557\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " Jul 07 13:39:50.085099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.084870 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-token\") pod \"dc7bdc96-fc50-4396-936a-f41cee5f3557\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " Jul 07 13:39:50.085099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.084906 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksh2r\" (UniqueName: \"kubernetes.io/projected/dc7bdc96-fc50-4396-936a-f41cee5f3557-kube-api-access-ksh2r\") pod \"dc7bdc96-fc50-4396-936a-f41cee5f3557\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " Jul 07 13:39:50.085099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.084935 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-workload-socket\") pod \"dc7bdc96-fc50-4396-936a-f41cee5f3557\" (UID: \"dc7bdc96-fc50-4396-936a-f41cee5f3557\") " Jul 07 13:39:50.085099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.085085 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc7bdc96-fc50-4396-936a-f41cee5f3557-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dc7bdc96-fc50-4396-936a-f41cee5f3557" (UID: "dc7bdc96-fc50-4396-936a-f41cee5f3557"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:39:50.085526 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.085429 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc7bdc96-fc50-4396-936a-f41cee5f3557-istiod-ca-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:50.085526 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.085470 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-data" (OuterVolumeSpecName: "istio-data") pod "dc7bdc96-fc50-4396-936a-f41cee5f3557" (UID: "dc7bdc96-fc50-4396-936a-f41cee5f3557"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:39:50.085929 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.085891 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dc7bdc96-fc50-4396-936a-f41cee5f3557" (UID: "dc7bdc96-fc50-4396-936a-f41cee5f3557"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:39:50.086121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.086095 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dc7bdc96-fc50-4396-936a-f41cee5f3557" (UID: "dc7bdc96-fc50-4396-936a-f41cee5f3557"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:39:50.086265 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.086236 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dc7bdc96-fc50-4396-936a-f41cee5f3557" (UID: "dc7bdc96-fc50-4396-936a-f41cee5f3557"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:39:50.088369 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.088327 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dc7bdc96-fc50-4396-936a-f41cee5f3557" (UID: "dc7bdc96-fc50-4396-936a-f41cee5f3557"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:39:50.088479 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.088462 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dc7bdc96-fc50-4396-936a-f41cee5f3557" (UID: "dc7bdc96-fc50-4396-936a-f41cee5f3557"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:39:50.088600 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.088560 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-token" (OuterVolumeSpecName: "istio-token") pod "dc7bdc96-fc50-4396-936a-f41cee5f3557" (UID: "dc7bdc96-fc50-4396-936a-f41cee5f3557"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:39:50.089044 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.089013 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc7bdc96-fc50-4396-936a-f41cee5f3557-kube-api-access-ksh2r" (OuterVolumeSpecName: "kube-api-access-ksh2r") pod "dc7bdc96-fc50-4396-936a-f41cee5f3557" (UID: "dc7bdc96-fc50-4396-936a-f41cee5f3557"). InnerVolumeSpecName "kube-api-access-ksh2r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:39:50.105625 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.105570 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb"] Jul 07 13:39:50.106279 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.106258 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc7bdc96-fc50-4396-936a-f41cee5f3557" containerName="istio-proxy" Jul 07 13:39:50.106353 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.106282 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc7bdc96-fc50-4396-936a-f41cee5f3557" containerName="istio-proxy" Jul 07 13:39:50.106433 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.106419 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc7bdc96-fc50-4396-936a-f41cee5f3557" containerName="istio-proxy" Jul 07 13:39:50.106488 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.106439 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc7bdc96-fc50-4396-936a-f41cee5f3557" containerName="istio-proxy" Jul 07 13:39:50.109277 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.109260 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.111959 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.111943 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-8c7f7fad-openshift-default-dockercfg-5hz9l\"" Jul 07 13:39:50.123556 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.123536 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb"] Jul 07 13:39:50.137475 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.137456 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:50.186142 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.186116 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:50.186142 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.186136 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-workload-certs\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:50.186142 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.186146 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-credential-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:50.186314 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.186156 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-envoy\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:50.186314 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.186165 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-podinfo\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:50.186314 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.186172 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc7bdc96-fc50-4396-936a-f41cee5f3557-istio-token\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:50.186314 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.186180 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksh2r\" (UniqueName: \"kubernetes.io/projected/dc7bdc96-fc50-4396-936a-f41cee5f3557-kube-api-access-ksh2r\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:50.186314 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.186189 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc7bdc96-fc50-4396-936a-f41cee5f3557-workload-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:39:50.287489 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.287394 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-credential-socket\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.287489 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.287441 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-envoy\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.287725 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.287495 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-data\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.287725 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.287561 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-workload-certs\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.287725 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.287618 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-podinfo\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.287725 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.287704 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-token\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.287877 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.287735 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istiod-ca-cert\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.287877 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.287759 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-workload-socket\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.287877 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.287781 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx7hl\" (UniqueName: \"kubernetes.io/projected/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-kube-api-access-kx7hl\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.389099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.389065 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-envoy\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.389235 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.389126 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-data\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.389235 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.389168 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-workload-certs\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.389235 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.389195 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-podinfo\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.389385 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.389330 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-token\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.389385 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.389375 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istiod-ca-cert\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.389489 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.389405 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-workload-socket\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.389489 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.389431 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx7hl\" (UniqueName: \"kubernetes.io/projected/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-kube-api-access-kx7hl\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.389489 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.389472 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-credential-socket\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.389690 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.389565 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-data\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.389690 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.389630 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-workload-certs\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.389863 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.389842 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-credential-socket\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.390189 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.390164 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istiod-ca-cert\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.390423 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.390403 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-workload-socket\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.391320 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.391296 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-envoy\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.391595 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.391563 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-podinfo\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.397828 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.397808 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-token\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.398508 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.398490 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx7hl\" (UniqueName: \"kubernetes.io/projected/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-kube-api-access-kx7hl\") pod \"e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.423095 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.423065 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:50.565786 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.565758 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb"] Jul 07 13:39:50.567954 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:39:50.567921 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c339a9d_edd1_4896_8cd1_57c6a72ea55c.slice/crio-4b5b5bab7801c037488a3df33db8a526d2425446832fd552d4c433adbaf84d0d WatchSource:0}: Error finding container 4b5b5bab7801c037488a3df33db8a526d2425446832fd552d4c433adbaf84d0d: Status 404 returned error can't find the container with id 4b5b5bab7801c037488a3df33db8a526d2425446832fd552d4c433adbaf84d0d Jul 07 13:39:50.570103 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.570066 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:39:50.570207 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.570137 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:39:50.570207 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.570168 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:39:50.724473 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.724442 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" event={"ID":"4c339a9d-edd1-4896-8cd1-57c6a72ea55c","Type":"ContainerStarted","Data":"a2363a99422bc0c3761da0658fce1c583c7d7b91b07469ba69ca3e3325f0ec17"} Jul 07 13:39:50.724932 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.724494 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" event={"ID":"4c339a9d-edd1-4896-8cd1-57c6a72ea55c","Type":"ContainerStarted","Data":"4b5b5bab7801c037488a3df33db8a526d2425446832fd552d4c433adbaf84d0d"} Jul 07 13:39:50.726270 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.726244 2568 generic.go:356] "Generic (PLEG): container finished" podID="dc7bdc96-fc50-4396-936a-f41cee5f3557" containerID="601bd14f5073a31fb7f03e012c60e577db78548e9bb86df72bd21240b4a6b7b6" exitCode=0 Jul 07 13:39:50.726388 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.726291 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" Jul 07 13:39:50.726388 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.726312 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" event={"ID":"dc7bdc96-fc50-4396-936a-f41cee5f3557","Type":"ContainerDied","Data":"601bd14f5073a31fb7f03e012c60e577db78548e9bb86df72bd21240b4a6b7b6"} Jul 07 13:39:50.726388 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.726360 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg" event={"ID":"dc7bdc96-fc50-4396-936a-f41cee5f3557","Type":"ContainerDied","Data":"71975019c2ecda287c0d97f39f7adb168b242732041dfdb3b8e6d2672013258a"} Jul 07 13:39:50.726486 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.726394 2568 scope.go:117] "RemoveContainer" containerID="601bd14f5073a31fb7f03e012c60e577db78548e9bb86df72bd21240b4a6b7b6" Jul 07 13:39:50.735496 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.735475 2568 scope.go:117] "RemoveContainer" containerID="fd323009582e5b6ef951e9314cd8170132320134cbe1312d23dc769623b5b8ee" Jul 07 13:39:50.744220 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.744197 2568 scope.go:117] "RemoveContainer" containerID="601bd14f5073a31fb7f03e012c60e577db78548e9bb86df72bd21240b4a6b7b6" Jul 07 13:39:50.744488 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:39:50.744472 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"601bd14f5073a31fb7f03e012c60e577db78548e9bb86df72bd21240b4a6b7b6\": container with ID starting with 601bd14f5073a31fb7f03e012c60e577db78548e9bb86df72bd21240b4a6b7b6 not found: ID does not exist" containerID="601bd14f5073a31fb7f03e012c60e577db78548e9bb86df72bd21240b4a6b7b6" Jul 07 13:39:50.744537 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.744497 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"601bd14f5073a31fb7f03e012c60e577db78548e9bb86df72bd21240b4a6b7b6"} err="failed to get container status \"601bd14f5073a31fb7f03e012c60e577db78548e9bb86df72bd21240b4a6b7b6\": rpc error: code = NotFound desc = could not find container \"601bd14f5073a31fb7f03e012c60e577db78548e9bb86df72bd21240b4a6b7b6\": container with ID starting with 601bd14f5073a31fb7f03e012c60e577db78548e9bb86df72bd21240b4a6b7b6 not found: ID does not exist" Jul 07 13:39:50.744537 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.744513 2568 scope.go:117] "RemoveContainer" containerID="fd323009582e5b6ef951e9314cd8170132320134cbe1312d23dc769623b5b8ee" Jul 07 13:39:50.744744 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:39:50.744730 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd323009582e5b6ef951e9314cd8170132320134cbe1312d23dc769623b5b8ee\": container with ID starting with fd323009582e5b6ef951e9314cd8170132320134cbe1312d23dc769623b5b8ee not found: ID does not exist" containerID="fd323009582e5b6ef951e9314cd8170132320134cbe1312d23dc769623b5b8ee" Jul 07 13:39:50.744795 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.744748 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd323009582e5b6ef951e9314cd8170132320134cbe1312d23dc769623b5b8ee"} err="failed to get container status \"fd323009582e5b6ef951e9314cd8170132320134cbe1312d23dc769623b5b8ee\": rpc error: code = NotFound desc = could not find container \"fd323009582e5b6ef951e9314cd8170132320134cbe1312d23dc769623b5b8ee\": container with ID starting with fd323009582e5b6ef951e9314cd8170132320134cbe1312d23dc769623b5b8ee not found: ID does not exist" Jul 07 13:39:50.746851 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.746805 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" podStartSLOduration=0.746789375 podStartE2EDuration="746.789375ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:39:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:39:50.746181109 +0000 UTC m=+1548.424742440" watchObservedRunningTime="2026-07-07 13:39:50.746789375 +0000 UTC m=+1548.425350706" Jul 07 13:39:50.765753 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.764182 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg"] Jul 07 13:39:50.770625 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.770561 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-de419fb7-openshift-default-7d788847bc-s2mlg"] Jul 07 13:39:50.897812 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.897782 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:50.974410 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.974368 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc7bdc96-fc50-4396-936a-f41cee5f3557" path="/var/lib/kubelet/pods/dc7bdc96-fc50-4396-936a-f41cee5f3557/volumes" Jul 07 13:39:50.997068 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:50.997042 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:51.282294 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:51.282259 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:51.391631 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:51.391604 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:51.423831 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:51.423794 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:51.428965 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:51.428938 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:51.732877 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:51.732843 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:51.733802 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:51.733779 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:39:52.612059 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:52.612027 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:52.713078 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:52.713047 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:54.220029 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:54.219999 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:55.752130 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:55.752100 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:55.921655 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:55.921619 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb"] Jul 07 13:39:55.921861 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:55.921833 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" podUID="4c339a9d-edd1-4896-8cd1-57c6a72ea55c" containerName="istio-proxy" containerID="cri-o://a2363a99422bc0c3761da0658fce1c583c7d7b91b07469ba69ca3e3325f0ec17" gracePeriod=30 Jul 07 13:39:55.941776 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:55.941746 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:56.112636 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:56.112552 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:56.180017 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:56.179988 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:39:57.487170 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:39:57.487138 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:01.678879 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.678817 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:40:01.709859 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.709832 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istiod-ca-cert\") pod \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " Jul 07 13:40:01.709991 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.709865 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-token\") pod \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " Jul 07 13:40:01.709991 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.709897 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-data\") pod \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " Jul 07 13:40:01.709991 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.709951 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-podinfo\") pod \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " Jul 07 13:40:01.710106 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.710027 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kx7hl\" (UniqueName: \"kubernetes.io/projected/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-kube-api-access-kx7hl\") pod \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " Jul 07 13:40:01.710106 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.710047 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-workload-certs\") pod \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " Jul 07 13:40:01.710106 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.710075 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-credential-socket\") pod \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " Jul 07 13:40:01.710346 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.710321 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4c339a9d-edd1-4896-8cd1-57c6a72ea55c" (UID: "4c339a9d-edd1-4896-8cd1-57c6a72ea55c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:40:01.710442 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.710334 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4c339a9d-edd1-4896-8cd1-57c6a72ea55c" (UID: "4c339a9d-edd1-4896-8cd1-57c6a72ea55c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:40:01.710485 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.710463 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-envoy\") pod \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " Jul 07 13:40:01.710562 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.710545 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-workload-socket\") pod \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\" (UID: \"4c339a9d-edd1-4896-8cd1-57c6a72ea55c\") " Jul 07 13:40:01.710641 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.710567 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4c339a9d-edd1-4896-8cd1-57c6a72ea55c" (UID: "4c339a9d-edd1-4896-8cd1-57c6a72ea55c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:40:01.710788 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.710769 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4c339a9d-edd1-4896-8cd1-57c6a72ea55c" (UID: "4c339a9d-edd1-4896-8cd1-57c6a72ea55c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:40:01.710862 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.710807 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-data" (OuterVolumeSpecName: "istio-data") pod "4c339a9d-edd1-4896-8cd1-57c6a72ea55c" (UID: "4c339a9d-edd1-4896-8cd1-57c6a72ea55c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:40:01.711012 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.710995 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-workload-certs\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:40:01.711084 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.711016 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-credential-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:40:01.711084 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.711031 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-workload-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:40:01.711084 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.711043 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istiod-ca-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:40:01.711084 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.711055 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:40:01.712710 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.712687 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-kube-api-access-kx7hl" (OuterVolumeSpecName: "kube-api-access-kx7hl") pod "4c339a9d-edd1-4896-8cd1-57c6a72ea55c" (UID: "4c339a9d-edd1-4896-8cd1-57c6a72ea55c"). InnerVolumeSpecName "kube-api-access-kx7hl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:40:01.712829 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.712711 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-token" (OuterVolumeSpecName: "istio-token") pod "4c339a9d-edd1-4896-8cd1-57c6a72ea55c" (UID: "4c339a9d-edd1-4896-8cd1-57c6a72ea55c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:40:01.712893 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.712863 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4c339a9d-edd1-4896-8cd1-57c6a72ea55c" (UID: "4c339a9d-edd1-4896-8cd1-57c6a72ea55c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:40:01.712992 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.712977 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4c339a9d-edd1-4896-8cd1-57c6a72ea55c" (UID: "4c339a9d-edd1-4896-8cd1-57c6a72ea55c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:40:01.793153 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.793117 2568 generic.go:356] "Generic (PLEG): container finished" podID="4c339a9d-edd1-4896-8cd1-57c6a72ea55c" containerID="a2363a99422bc0c3761da0658fce1c583c7d7b91b07469ba69ca3e3325f0ec17" exitCode=0 Jul 07 13:40:01.793328 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.793173 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" Jul 07 13:40:01.793328 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.793208 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" event={"ID":"4c339a9d-edd1-4896-8cd1-57c6a72ea55c","Type":"ContainerDied","Data":"a2363a99422bc0c3761da0658fce1c583c7d7b91b07469ba69ca3e3325f0ec17"} Jul 07 13:40:01.793328 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.793258 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb" event={"ID":"4c339a9d-edd1-4896-8cd1-57c6a72ea55c","Type":"ContainerDied","Data":"4b5b5bab7801c037488a3df33db8a526d2425446832fd552d4c433adbaf84d0d"} Jul 07 13:40:01.793328 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.793278 2568 scope.go:117] "RemoveContainer" containerID="a2363a99422bc0c3761da0658fce1c583c7d7b91b07469ba69ca3e3325f0ec17" Jul 07 13:40:01.802614 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.802597 2568 scope.go:117] "RemoveContainer" containerID="a2363a99422bc0c3761da0658fce1c583c7d7b91b07469ba69ca3e3325f0ec17" Jul 07 13:40:01.802853 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:40:01.802835 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2363a99422bc0c3761da0658fce1c583c7d7b91b07469ba69ca3e3325f0ec17\": container with ID starting with a2363a99422bc0c3761da0658fce1c583c7d7b91b07469ba69ca3e3325f0ec17 not found: ID does not exist" containerID="a2363a99422bc0c3761da0658fce1c583c7d7b91b07469ba69ca3e3325f0ec17" Jul 07 13:40:01.802906 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.802861 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2363a99422bc0c3761da0658fce1c583c7d7b91b07469ba69ca3e3325f0ec17"} err="failed to get container status \"a2363a99422bc0c3761da0658fce1c583c7d7b91b07469ba69ca3e3325f0ec17\": rpc error: code = NotFound desc = could not find container \"a2363a99422bc0c3761da0658fce1c583c7d7b91b07469ba69ca3e3325f0ec17\": container with ID starting with a2363a99422bc0c3761da0658fce1c583c7d7b91b07469ba69ca3e3325f0ec17 not found: ID does not exist" Jul 07 13:40:01.812511 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.812485 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kx7hl\" (UniqueName: \"kubernetes.io/projected/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-kube-api-access-kx7hl\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:40:01.812667 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.812515 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-envoy\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:40:01.812667 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.812530 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-token\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:40:01.812667 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.812542 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c339a9d-edd1-4896-8cd1-57c6a72ea55c-istio-podinfo\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:40:01.816842 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.816818 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb"] Jul 07 13:40:01.821090 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:01.821067 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-8c7f7fad-openshift-default-7f477bcdc5-jkfmb"] Jul 07 13:40:02.284313 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:02.284276 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:02.355441 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:02.355416 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:02.898926 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:02.898898 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:02.971682 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:02.971654 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c339a9d-edd1-4896-8cd1-57c6a72ea55c" path="/var/lib/kubelet/pods/4c339a9d-edd1-4896-8cd1-57c6a72ea55c/volumes" Jul 07 13:40:03.095071 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:03.095043 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:04.598406 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:04.598375 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:10.404429 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:10.404398 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:10.475742 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:10.475710 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:10.572772 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:10.572740 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:10.674387 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:10.674359 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:11.204837 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:11.204804 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:28.470512 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:28.470476 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:37.116672 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:37.116634 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:43.105077 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:43.105044 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:43.212483 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:43.212455 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:43.305191 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:43.305163 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:40:59.223975 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:40:59.223893 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:05.312377 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:05.312345 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:05.418163 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:05.418130 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:05.513478 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:05.513440 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:05.893537 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:05.893507 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:16.077245 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:16.077203 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:16.914461 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:16.914426 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:22.955051 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:22.955016 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:23.026724 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:23.026698 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:23.114809 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:23.114781 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:23.215452 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:23.215362 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:24.026709 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:24.026679 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:24.119124 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:24.119098 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:24.221884 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:24.221853 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:25.816871 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:25.816834 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:33.733053 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:33.733020 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:33.836398 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:33.836371 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:38.415006 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:38.414974 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:38.514937 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:38.514908 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:38.654219 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:38.654185 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:38.960126 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:38.960094 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz"] Jul 07 13:41:38.960533 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:38.960520 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc7bdc96-fc50-4396-936a-f41cee5f3557" containerName="istio-proxy" Jul 07 13:41:38.960632 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:38.960534 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc7bdc96-fc50-4396-936a-f41cee5f3557" containerName="istio-proxy" Jul 07 13:41:38.960632 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:38.960560 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c339a9d-edd1-4896-8cd1-57c6a72ea55c" containerName="istio-proxy" Jul 07 13:41:38.960632 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:38.960566 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c339a9d-edd1-4896-8cd1-57c6a72ea55c" containerName="istio-proxy" Jul 07 13:41:38.960739 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:38.960652 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c339a9d-edd1-4896-8cd1-57c6a72ea55c" containerName="istio-proxy" Jul 07 13:41:38.962769 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:38.962754 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:38.966717 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:38.966695 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-bc796f0c-openshift-default-dockercfg-cfhkz\"" Jul 07 13:41:38.985384 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:38.985357 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:38.985635 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:38.985617 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz"] Jul 07 13:41:39.058378 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.058346 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-token\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.058563 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.058386 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istiod-ca-cert\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.058563 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.058414 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-credential-socket\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.058563 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.058473 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-envoy\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.058563 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.058528 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77k8n\" (UniqueName: \"kubernetes.io/projected/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-kube-api-access-77k8n\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.058810 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.058612 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-workload-certs\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.058810 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.058660 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-data\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.058810 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.058718 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-workload-socket\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.058810 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.058750 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-podinfo\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.142399 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.142371 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:39.159931 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.159903 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-workload-certs\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.160082 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.159937 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-data\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.160082 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.159973 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-workload-socket\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.160082 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.159994 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-podinfo\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.160082 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.160036 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-token\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.160082 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.160067 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istiod-ca-cert\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.160347 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.160112 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-credential-socket\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.160347 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.160134 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-envoy\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.160347 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.160183 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77k8n\" (UniqueName: \"kubernetes.io/projected/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-kube-api-access-77k8n\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.160347 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.160301 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-workload-certs\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.160540 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.160372 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-data\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.160540 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.160443 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-credential-socket\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.160688 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.160672 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-workload-socket\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.160873 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.160851 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istiod-ca-cert\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.162426 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.162399 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-envoy\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.162550 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.162514 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-podinfo\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.173428 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.173406 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77k8n\" (UniqueName: \"kubernetes.io/projected/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-kube-api-access-77k8n\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.179547 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.179523 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-token\") pod \"e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.274128 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.274045 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:39.413547 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.413518 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz"] Jul 07 13:41:39.414482 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:41:39.414455 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f9dc2e3_49d6_4582_b514_3cfc3b3ae91e.slice/crio-8d9fdb0efacff20402b432907af8a7144073cdc3cf48564f252981748659a3d4 WatchSource:0}: Error finding container 8d9fdb0efacff20402b432907af8a7144073cdc3cf48564f252981748659a3d4: Status 404 returned error can't find the container with id 8d9fdb0efacff20402b432907af8a7144073cdc3cf48564f252981748659a3d4 Jul 07 13:41:39.416676 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.416632 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:41:39.416953 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.416705 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:41:39.416953 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:39.416754 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:41:40.292066 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:40.292030 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" event={"ID":"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e","Type":"ContainerStarted","Data":"86b08535a35127ec5cf301e2fe854207ee24dedf5e2a3d1adbed5e46fddabfe5"} Jul 07 13:41:40.292238 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:40.292081 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" event={"ID":"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e","Type":"ContainerStarted","Data":"8d9fdb0efacff20402b432907af8a7144073cdc3cf48564f252981748659a3d4"} Jul 07 13:41:40.318032 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:40.317985 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" podStartSLOduration=2.317970138 podStartE2EDuration="2.317970138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:41:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:41:40.315337433 +0000 UTC m=+1657.993898764" watchObservedRunningTime="2026-07-07 13:41:40.317970138 +0000 UTC m=+1657.996531531" Jul 07 13:41:41.274471 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:41.274440 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:41.279328 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:41.279307 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:41.297557 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:41.297521 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:41.298159 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:41.298138 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:42.339371 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:42.339336 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:42.537360 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:42.537324 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:42.688147 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:42.688118 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:42.827112 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:42.827085 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:43.304553 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:43.304520 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:43.404636 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:43.404591 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:46.769296 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:46.769257 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:47.503267 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:47.503230 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:47.631907 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:47.631872 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:47.942166 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:47.942127 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:53.781781 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:53.781743 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:53.850968 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:53.850941 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:54.189890 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:54.189852 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:54.500338 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:54.500250 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:54.573622 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:54.573589 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:41:56.299276 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:56.299240 2568 patch_prober.go:28] interesting pod/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 07 13:41:56.299713 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:56.299295 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" podUID="3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 07 13:41:56.380794 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:56.380706 2568 generic.go:356] "Generic (PLEG): container finished" podID="3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" containerID="86b08535a35127ec5cf301e2fe854207ee24dedf5e2a3d1adbed5e46fddabfe5" exitCode=0 Jul 07 13:41:56.380794 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:56.380776 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" event={"ID":"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e","Type":"ContainerDied","Data":"86b08535a35127ec5cf301e2fe854207ee24dedf5e2a3d1adbed5e46fddabfe5"} Jul 07 13:41:56.381230 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:56.381215 2568 scope.go:117] "RemoveContainer" containerID="86b08535a35127ec5cf301e2fe854207ee24dedf5e2a3d1adbed5e46fddabfe5" Jul 07 13:41:56.382251 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:56.382213 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:41:56.382380 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:56.382300 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:41:56.382380 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:56.382350 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:41:57.388248 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:57.388217 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" event={"ID":"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e","Type":"ContainerStarted","Data":"0f2691fe0c288319bab132a6c5b388179cc9e46c6f5e66a40ef59186804ba0c3"} Jul 07 13:41:58.274492 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:58.274455 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:41:58.274770 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:58.274748 2568 patch_prober.go:28] interesting pod/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 07 13:41:58.274827 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:58.274805 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" podUID="3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 07 13:41:59.275414 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:59.275374 2568 patch_prober.go:28] interesting pod/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 07 13:41:59.275856 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:41:59.275436 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" podUID="3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 07 13:42:00.275147 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:00.275112 2568 patch_prober.go:28] interesting pod/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 07 13:42:00.275312 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:00.275190 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" podUID="3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 07 13:42:00.298865 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:00.298830 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:00.365939 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:00.365901 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:00.607068 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:00.606972 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz"] Jul 07 13:42:00.607241 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:00.607212 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" podUID="3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" containerName="istio-proxy" containerID="cri-o://0f2691fe0c288319bab132a6c5b388179cc9e46c6f5e66a40ef59186804ba0c3" gracePeriod=30 Jul 07 13:42:00.646007 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:00.645978 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:00.820915 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:00.820884 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:00.968609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:00.968560 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:01.068460 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:01.068428 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:01.176140 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:01.176110 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:06.857718 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.857690 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:42:06.949940 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.949906 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-envoy\") pod \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " Jul 07 13:42:06.950123 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.949983 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-token\") pod \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " Jul 07 13:42:06.950123 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.950001 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-workload-socket\") pod \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " Jul 07 13:42:06.950123 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.950048 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-workload-certs\") pod \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " Jul 07 13:42:06.950123 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.950102 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istiod-ca-cert\") pod \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " Jul 07 13:42:06.950123 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.950123 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77k8n\" (UniqueName: \"kubernetes.io/projected/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-kube-api-access-77k8n\") pod \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " Jul 07 13:42:06.950390 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.950156 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-podinfo\") pod \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " Jul 07 13:42:06.950390 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.950192 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-credential-socket\") pod \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " Jul 07 13:42:06.950390 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.950233 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-data\") pod \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\" (UID: \"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e\") " Jul 07 13:42:06.950390 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.950322 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" (UID: "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:06.950621 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.950554 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" (UID: "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:06.950677 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.950622 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" (UID: "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:42:06.950806 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.950781 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" (UID: "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:06.950874 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.950858 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istiod-ca-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:42:06.950917 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.950878 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-credential-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:42:06.950917 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.950893 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-workload-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:42:06.951567 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.951542 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-data" (OuterVolumeSpecName: "istio-data") pod "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" (UID: "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:06.952297 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.952277 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-token" (OuterVolumeSpecName: "istio-token") pod "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" (UID: "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:42:06.952502 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.952474 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-kube-api-access-77k8n" (OuterVolumeSpecName: "kube-api-access-77k8n") pod "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" (UID: "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e"). InnerVolumeSpecName "kube-api-access-77k8n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:42:06.952653 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.952607 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" (UID: "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:06.952653 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:06.952609 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" (UID: "3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:42:07.052022 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.051994 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77k8n\" (UniqueName: \"kubernetes.io/projected/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-kube-api-access-77k8n\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:42:07.052022 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.052018 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-podinfo\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:42:07.052022 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.052027 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:42:07.052227 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.052036 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-envoy\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:42:07.052227 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.052044 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-istio-token\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:42:07.052227 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.052052 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e-workload-certs\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:42:07.155759 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.155730 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:07.258249 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.258093 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:07.359942 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.359915 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:07.443001 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.442972 2568 generic.go:356] "Generic (PLEG): container finished" podID="3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" containerID="0f2691fe0c288319bab132a6c5b388179cc9e46c6f5e66a40ef59186804ba0c3" exitCode=0 Jul 07 13:42:07.443173 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.443029 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" Jul 07 13:42:07.443173 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.443053 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" event={"ID":"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e","Type":"ContainerDied","Data":"0f2691fe0c288319bab132a6c5b388179cc9e46c6f5e66a40ef59186804ba0c3"} Jul 07 13:42:07.443173 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.443105 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz" event={"ID":"3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e","Type":"ContainerDied","Data":"8d9fdb0efacff20402b432907af8a7144073cdc3cf48564f252981748659a3d4"} Jul 07 13:42:07.443173 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.443130 2568 scope.go:117] "RemoveContainer" containerID="0f2691fe0c288319bab132a6c5b388179cc9e46c6f5e66a40ef59186804ba0c3" Jul 07 13:42:07.452551 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.452535 2568 scope.go:117] "RemoveContainer" containerID="86b08535a35127ec5cf301e2fe854207ee24dedf5e2a3d1adbed5e46fddabfe5" Jul 07 13:42:07.460545 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.460523 2568 scope.go:117] "RemoveContainer" containerID="0f2691fe0c288319bab132a6c5b388179cc9e46c6f5e66a40ef59186804ba0c3" Jul 07 13:42:07.460827 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:42:07.460808 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f2691fe0c288319bab132a6c5b388179cc9e46c6f5e66a40ef59186804ba0c3\": container with ID starting with 0f2691fe0c288319bab132a6c5b388179cc9e46c6f5e66a40ef59186804ba0c3 not found: ID does not exist" containerID="0f2691fe0c288319bab132a6c5b388179cc9e46c6f5e66a40ef59186804ba0c3" Jul 07 13:42:07.460912 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.460838 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f2691fe0c288319bab132a6c5b388179cc9e46c6f5e66a40ef59186804ba0c3"} err="failed to get container status \"0f2691fe0c288319bab132a6c5b388179cc9e46c6f5e66a40ef59186804ba0c3\": rpc error: code = NotFound desc = could not find container \"0f2691fe0c288319bab132a6c5b388179cc9e46c6f5e66a40ef59186804ba0c3\": container with ID starting with 0f2691fe0c288319bab132a6c5b388179cc9e46c6f5e66a40ef59186804ba0c3 not found: ID does not exist" Jul 07 13:42:07.460912 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.460863 2568 scope.go:117] "RemoveContainer" containerID="86b08535a35127ec5cf301e2fe854207ee24dedf5e2a3d1adbed5e46fddabfe5" Jul 07 13:42:07.461132 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:42:07.461116 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86b08535a35127ec5cf301e2fe854207ee24dedf5e2a3d1adbed5e46fddabfe5\": container with ID starting with 86b08535a35127ec5cf301e2fe854207ee24dedf5e2a3d1adbed5e46fddabfe5 not found: ID does not exist" containerID="86b08535a35127ec5cf301e2fe854207ee24dedf5e2a3d1adbed5e46fddabfe5" Jul 07 13:42:07.461172 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.461137 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86b08535a35127ec5cf301e2fe854207ee24dedf5e2a3d1adbed5e46fddabfe5"} err="failed to get container status \"86b08535a35127ec5cf301e2fe854207ee24dedf5e2a3d1adbed5e46fddabfe5\": rpc error: code = NotFound desc = could not find container \"86b08535a35127ec5cf301e2fe854207ee24dedf5e2a3d1adbed5e46fddabfe5\": container with ID starting with 86b08535a35127ec5cf301e2fe854207ee24dedf5e2a3d1adbed5e46fddabfe5 not found: ID does not exist" Jul 07 13:42:07.462149 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.462131 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz"] Jul 07 13:42:07.468472 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.468453 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-bc796f0c-openshift-default-5bb6bf487f-n2fvz"] Jul 07 13:42:07.487534 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:07.487515 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:08.970802 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:08.970772 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" path="/var/lib/kubelet/pods/3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e/volumes" Jul 07 13:42:13.942239 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:13.942204 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:14.110203 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:14.110164 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:21.037674 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.037638 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z"] Jul 07 13:42:21.038128 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.038096 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" containerName="istio-proxy" Jul 07 13:42:21.038128 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.038108 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" containerName="istio-proxy" Jul 07 13:42:21.038200 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.038188 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" containerName="istio-proxy" Jul 07 13:42:21.038200 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.038198 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" containerName="istio-proxy" Jul 07 13:42:21.041348 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.041332 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.044459 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.044438 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8880b7f7-openshift-default-dockercfg-9szjj\"" Jul 07 13:42:21.053437 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.053414 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z"] Jul 07 13:42:21.177558 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.177525 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:21.199142 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.199110 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-workload-certs\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.199301 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.199175 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-token\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.199301 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.199252 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-envoy\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.199301 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.199283 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-credential-socket\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.199416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.199352 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrlr2\" (UniqueName: \"kubernetes.io/projected/2af35c88-ad2e-49ef-bb98-ace302681ec6-kube-api-access-rrlr2\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.199416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.199391 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-data\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.199416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.199407 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-podinfo\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.199558 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.199501 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-workload-socket\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.199634 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.199554 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af35c88-ad2e-49ef-bb98-ace302681ec6-istiod-ca-cert\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.278746 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.278712 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:21.300278 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.300218 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-credential-socket\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.300278 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.300265 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrlr2\" (UniqueName: \"kubernetes.io/projected/2af35c88-ad2e-49ef-bb98-ace302681ec6-kube-api-access-rrlr2\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.300390 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.300296 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-data\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.300390 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.300313 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-podinfo\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.300390 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.300341 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-workload-socket\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.300390 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.300381 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af35c88-ad2e-49ef-bb98-ace302681ec6-istiod-ca-cert\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.300561 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.300422 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-workload-certs\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.300561 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.300483 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-token\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.300561 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.300529 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-envoy\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.300727 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.300658 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-data\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.300727 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.300656 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-credential-socket\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.300829 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.300739 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-workload-socket\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.300950 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.300930 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-workload-certs\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.301210 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.301187 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af35c88-ad2e-49ef-bb98-ace302681ec6-istiod-ca-cert\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.302677 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.302652 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-envoy\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.303061 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.303037 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-podinfo\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.309503 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.309483 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrlr2\" (UniqueName: \"kubernetes.io/projected/2af35c88-ad2e-49ef-bb98-ace302681ec6-kube-api-access-rrlr2\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.311325 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.311303 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-token\") pod \"e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.352320 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.352296 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:21.488762 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.488684 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z"] Jul 07 13:42:21.491156 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:42:21.491124 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af35c88_ad2e_49ef_bb98_ace302681ec6.slice/crio-160b3db41335d6fdef288f666805a3f1c0c99b079a8791c2b7f08c6b136c20f2 WatchSource:0}: Error finding container 160b3db41335d6fdef288f666805a3f1c0c99b079a8791c2b7f08c6b136c20f2: Status 404 returned error can't find the container with id 160b3db41335d6fdef288f666805a3f1c0c99b079a8791c2b7f08c6b136c20f2 Jul 07 13:42:21.493287 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.493254 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:42:21.493346 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.493319 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:42:21.493390 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.493346 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:42:21.517091 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:21.517068 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" event={"ID":"2af35c88-ad2e-49ef-bb98-ace302681ec6","Type":"ContainerStarted","Data":"160b3db41335d6fdef288f666805a3f1c0c99b079a8791c2b7f08c6b136c20f2"} Jul 07 13:42:22.523906 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:22.523864 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" event={"ID":"2af35c88-ad2e-49ef-bb98-ace302681ec6","Type":"ContainerStarted","Data":"cf034ed837d4afa7d773a2e2c961a067477a51fdd6fca571c329cec4be218f22"} Jul 07 13:42:22.548342 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:22.548287 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" podStartSLOduration=1.548270029 podStartE2EDuration="1.548270029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:42:22.54712638 +0000 UTC m=+1700.225687712" watchObservedRunningTime="2026-07-07 13:42:22.548270029 +0000 UTC m=+1700.226831360" Jul 07 13:42:23.353347 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:23.353307 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:23.358160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:23.358134 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:23.529040 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:23.529001 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:23.530097 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:23.530080 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:24.473981 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:24.473892 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:24.578389 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:24.578356 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:24.673337 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:24.673311 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:25.137457 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:25.137424 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:25.240254 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:25.240225 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:27.284000 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:27.283969 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:27.386077 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:27.386041 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:29.000035 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:29.000004 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:37.610318 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:37.610237 2568 generic.go:356] "Generic (PLEG): container finished" podID="2af35c88-ad2e-49ef-bb98-ace302681ec6" containerID="cf034ed837d4afa7d773a2e2c961a067477a51fdd6fca571c329cec4be218f22" exitCode=0 Jul 07 13:42:37.610750 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:37.610312 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" event={"ID":"2af35c88-ad2e-49ef-bb98-ace302681ec6","Type":"ContainerDied","Data":"cf034ed837d4afa7d773a2e2c961a067477a51fdd6fca571c329cec4be218f22"} Jul 07 13:42:37.610802 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:37.610789 2568 scope.go:117] "RemoveContainer" containerID="cf034ed837d4afa7d773a2e2c961a067477a51fdd6fca571c329cec4be218f22" Jul 07 13:42:37.611838 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:37.611802 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:42:37.611955 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:37.611891 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:42:37.611955 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:37.611941 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:42:38.530065 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:38.530031 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:38.617723 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:38.617688 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" event={"ID":"2af35c88-ad2e-49ef-bb98-ace302681ec6","Type":"ContainerStarted","Data":"b93190e068f697f9202889cd140f74f20a948857d80c6d27cdac53f4863d2c54"} Jul 07 13:42:39.353502 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:39.353462 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:39.354755 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:39.354728 2568 patch_prober.go:28] interesting pod/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 07 13:42:39.354895 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:39.354796 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" podUID="2af35c88-ad2e-49ef-bb98-ace302681ec6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 07 13:42:40.353022 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:40.352982 2568 patch_prober.go:28] interesting pod/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 07 13:42:40.353517 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:40.353067 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" podUID="2af35c88-ad2e-49ef-bb98-ace302681ec6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 07 13:42:41.353818 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:41.353772 2568 patch_prober.go:28] interesting pod/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 07 13:42:41.354310 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:41.353832 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" podUID="2af35c88-ad2e-49ef-bb98-ace302681ec6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 07 13:42:42.353517 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:42.353480 2568 patch_prober.go:28] interesting pod/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 07 13:42:42.353725 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:42.353548 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" podUID="2af35c88-ad2e-49ef-bb98-ace302681ec6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 07 13:42:43.356671 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:43.356640 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:43.357037 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:43.357000 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:43.357663 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:43.357648 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:42:46.550364 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:46.550337 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:42:48.036923 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:42:48.036890 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:43:06.764750 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:43:06.764714 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:43:07.123121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:43:07.123035 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:43:12.789827 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:43:12.789797 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:43:17.819662 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:43:17.819627 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:43:32.137349 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:43:32.137312 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:43:55.403754 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:43:55.403671 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:43:55.857986 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:43:55.857953 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:02.784307 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:02.784278 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:09.521286 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:09.521249 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:09.658312 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:09.658280 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:09.792045 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:09.791960 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z"] Jul 07 13:44:09.792185 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:09.792133 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" podUID="2af35c88-ad2e-49ef-bb98-ace302681ec6" containerName="istio-proxy" containerID="cri-o://b93190e068f697f9202889cd140f74f20a948857d80c6d27cdac53f4863d2c54" gracePeriod=30 Jul 07 13:44:09.853110 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:09.853079 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:10.086363 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:10.086279 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:11.209112 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:11.209079 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:11.363939 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:11.363910 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:12.273799 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:12.273766 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:12.455216 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:12.455181 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:13.079416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:13.079377 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:13.359250 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:13.359168 2568 patch_prober.go:28] interesting pod/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 13:44:13.359250 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:13.359228 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" podUID="2af35c88-ad2e-49ef-bb98-ace302681ec6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 13:44:13.427914 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:13.427889 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:14.140125 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:14.140098 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:15.046862 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.046832 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:44:15.123138 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.123040 2568 generic.go:356] "Generic (PLEG): container finished" podID="2af35c88-ad2e-49ef-bb98-ace302681ec6" containerID="b93190e068f697f9202889cd140f74f20a948857d80c6d27cdac53f4863d2c54" exitCode=0 Jul 07 13:44:15.123138 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.123105 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" Jul 07 13:44:15.123368 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.123161 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" event={"ID":"2af35c88-ad2e-49ef-bb98-ace302681ec6","Type":"ContainerDied","Data":"b93190e068f697f9202889cd140f74f20a948857d80c6d27cdac53f4863d2c54"} Jul 07 13:44:15.123368 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.123206 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z" event={"ID":"2af35c88-ad2e-49ef-bb98-ace302681ec6","Type":"ContainerDied","Data":"160b3db41335d6fdef288f666805a3f1c0c99b079a8791c2b7f08c6b136c20f2"} Jul 07 13:44:15.123368 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.123234 2568 scope.go:117] "RemoveContainer" containerID="b93190e068f697f9202889cd140f74f20a948857d80c6d27cdac53f4863d2c54" Jul 07 13:44:15.133152 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.133124 2568 scope.go:117] "RemoveContainer" containerID="cf034ed837d4afa7d773a2e2c961a067477a51fdd6fca571c329cec4be218f22" Jul 07 13:44:15.141077 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.141059 2568 scope.go:117] "RemoveContainer" containerID="b93190e068f697f9202889cd140f74f20a948857d80c6d27cdac53f4863d2c54" Jul 07 13:44:15.141318 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:44:15.141295 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b93190e068f697f9202889cd140f74f20a948857d80c6d27cdac53f4863d2c54\": container with ID starting with b93190e068f697f9202889cd140f74f20a948857d80c6d27cdac53f4863d2c54 not found: ID does not exist" containerID="b93190e068f697f9202889cd140f74f20a948857d80c6d27cdac53f4863d2c54" Jul 07 13:44:15.141377 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.141330 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b93190e068f697f9202889cd140f74f20a948857d80c6d27cdac53f4863d2c54"} err="failed to get container status \"b93190e068f697f9202889cd140f74f20a948857d80c6d27cdac53f4863d2c54\": rpc error: code = NotFound desc = could not find container \"b93190e068f697f9202889cd140f74f20a948857d80c6d27cdac53f4863d2c54\": container with ID starting with b93190e068f697f9202889cd140f74f20a948857d80c6d27cdac53f4863d2c54 not found: ID does not exist" Jul 07 13:44:15.141377 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.141348 2568 scope.go:117] "RemoveContainer" containerID="cf034ed837d4afa7d773a2e2c961a067477a51fdd6fca571c329cec4be218f22" Jul 07 13:44:15.141616 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:44:15.141599 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf034ed837d4afa7d773a2e2c961a067477a51fdd6fca571c329cec4be218f22\": container with ID starting with cf034ed837d4afa7d773a2e2c961a067477a51fdd6fca571c329cec4be218f22 not found: ID does not exist" containerID="cf034ed837d4afa7d773a2e2c961a067477a51fdd6fca571c329cec4be218f22" Jul 07 13:44:15.141693 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.141622 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf034ed837d4afa7d773a2e2c961a067477a51fdd6fca571c329cec4be218f22"} err="failed to get container status \"cf034ed837d4afa7d773a2e2c961a067477a51fdd6fca571c329cec4be218f22\": rpc error: code = NotFound desc = could not find container \"cf034ed837d4afa7d773a2e2c961a067477a51fdd6fca571c329cec4be218f22\": container with ID starting with cf034ed837d4afa7d773a2e2c961a067477a51fdd6fca571c329cec4be218f22 not found: ID does not exist" Jul 07 13:44:15.147926 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.147909 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-data\") pod \"2af35c88-ad2e-49ef-bb98-ace302681ec6\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " Jul 07 13:44:15.147988 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.147955 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-workload-certs\") pod \"2af35c88-ad2e-49ef-bb98-ace302681ec6\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " Jul 07 13:44:15.148043 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.147988 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-envoy\") pod \"2af35c88-ad2e-49ef-bb98-ace302681ec6\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " Jul 07 13:44:15.148043 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.148007 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-podinfo\") pod \"2af35c88-ad2e-49ef-bb98-ace302681ec6\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " Jul 07 13:44:15.148043 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.148032 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-token\") pod \"2af35c88-ad2e-49ef-bb98-ace302681ec6\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " Jul 07 13:44:15.148191 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.148174 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-workload-socket\") pod \"2af35c88-ad2e-49ef-bb98-ace302681ec6\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " Jul 07 13:44:15.148246 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.148233 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrlr2\" (UniqueName: \"kubernetes.io/projected/2af35c88-ad2e-49ef-bb98-ace302681ec6-kube-api-access-rrlr2\") pod \"2af35c88-ad2e-49ef-bb98-ace302681ec6\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " Jul 07 13:44:15.148299 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.148269 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-credential-socket\") pod \"2af35c88-ad2e-49ef-bb98-ace302681ec6\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " Jul 07 13:44:15.148352 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.148300 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2af35c88-ad2e-49ef-bb98-ace302681ec6" (UID: "2af35c88-ad2e-49ef-bb98-ace302681ec6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:44:15.148403 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.148354 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af35c88-ad2e-49ef-bb98-ace302681ec6-istiod-ca-cert\") pod \"2af35c88-ad2e-49ef-bb98-ace302681ec6\" (UID: \"2af35c88-ad2e-49ef-bb98-ace302681ec6\") " Jul 07 13:44:15.148900 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.148542 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2af35c88-ad2e-49ef-bb98-ace302681ec6" (UID: "2af35c88-ad2e-49ef-bb98-ace302681ec6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:44:15.148900 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.148695 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2af35c88-ad2e-49ef-bb98-ace302681ec6" (UID: "2af35c88-ad2e-49ef-bb98-ace302681ec6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:44:15.148900 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.148778 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-data" (OuterVolumeSpecName: "istio-data") pod "2af35c88-ad2e-49ef-bb98-ace302681ec6" (UID: "2af35c88-ad2e-49ef-bb98-ace302681ec6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:44:15.148900 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.148843 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2af35c88-ad2e-49ef-bb98-ace302681ec6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2af35c88-ad2e-49ef-bb98-ace302681ec6" (UID: "2af35c88-ad2e-49ef-bb98-ace302681ec6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:44:15.149188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.148980 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-workload-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:44:15.149188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.149004 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-credential-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:44:15.149188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.149014 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af35c88-ad2e-49ef-bb98-ace302681ec6-istiod-ca-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:44:15.149188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.149022 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:44:15.149188 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.149030 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-workload-certs\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:44:15.150197 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.150173 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-token" (OuterVolumeSpecName: "istio-token") pod "2af35c88-ad2e-49ef-bb98-ace302681ec6" (UID: "2af35c88-ad2e-49ef-bb98-ace302681ec6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:44:15.150277 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.150220 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2af35c88-ad2e-49ef-bb98-ace302681ec6" (UID: "2af35c88-ad2e-49ef-bb98-ace302681ec6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:44:15.150406 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.150393 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2af35c88-ad2e-49ef-bb98-ace302681ec6" (UID: "2af35c88-ad2e-49ef-bb98-ace302681ec6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:44:15.150482 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.150468 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2af35c88-ad2e-49ef-bb98-ace302681ec6-kube-api-access-rrlr2" (OuterVolumeSpecName: "kube-api-access-rrlr2") pod "2af35c88-ad2e-49ef-bb98-ace302681ec6" (UID: "2af35c88-ad2e-49ef-bb98-ace302681ec6"). InnerVolumeSpecName "kube-api-access-rrlr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:44:15.249779 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.249748 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrlr2\" (UniqueName: \"kubernetes.io/projected/2af35c88-ad2e-49ef-bb98-ace302681ec6-kube-api-access-rrlr2\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:44:15.249779 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.249773 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-envoy\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:44:15.249779 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.249782 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-podinfo\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:44:15.249975 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.249790 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af35c88-ad2e-49ef-bb98-ace302681ec6-istio-token\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:44:15.514538 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.514506 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z"] Jul 07 13:44:15.524433 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:15.524406 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8880b7f7-openshift-default-6759988755-7nx4z"] Jul 07 13:44:16.971097 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:16.971062 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2af35c88-ad2e-49ef-bb98-ace302681ec6" path="/var/lib/kubelet/pods/2af35c88-ad2e-49ef-bb98-ace302681ec6/volumes" Jul 07 13:44:21.103602 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:21.103531 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:21.254391 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:21.254359 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:21.870599 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:21.870543 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:29.141308 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:29.141265 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:29.435262 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:29.435229 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:39.939735 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:39.939701 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:44:40.287868 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:44:40.287771 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:01.619900 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:01.619860 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:11.563204 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:11.563169 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:11.719140 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:11.719111 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:12.106622 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.106565 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:12.385909 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.385820 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4"] Jul 07 13:45:12.386356 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.386337 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2af35c88-ad2e-49ef-bb98-ace302681ec6" containerName="istio-proxy" Jul 07 13:45:12.386447 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.386359 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2af35c88-ad2e-49ef-bb98-ace302681ec6" containerName="istio-proxy" Jul 07 13:45:12.386447 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.386376 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" containerName="istio-proxy" Jul 07 13:45:12.386447 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.386383 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f9dc2e3-49d6-4582-b514-3cfc3b3ae91e" containerName="istio-proxy" Jul 07 13:45:12.386625 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.386508 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2af35c88-ad2e-49ef-bb98-ace302681ec6" containerName="istio-proxy" Jul 07 13:45:12.388808 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.388788 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.392558 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.392535 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-055c02-openshift-default-dockercfg-kq98r\"" Jul 07 13:45:12.488884 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.488846 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:12.512528 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.512500 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/828c54da-1615-4817-9988-d940371838b6-istio-token\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.512734 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.512535 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-credential-socket\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.512734 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.512665 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/828c54da-1615-4817-9988-d940371838b6-istiod-ca-cert\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.512734 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.512729 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frc4m\" (UniqueName: \"kubernetes.io/projected/828c54da-1615-4817-9988-d940371838b6-kube-api-access-frc4m\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.512927 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.512796 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-workload-socket\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.512927 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.512848 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/828c54da-1615-4817-9988-d940371838b6-istio-podinfo\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.512927 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.512905 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-workload-certs\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.513075 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.512929 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-istio-envoy\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.513075 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.512961 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-istio-data\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.586975 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.586938 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4"] Jul 07 13:45:12.614323 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.614294 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/828c54da-1615-4817-9988-d940371838b6-istio-podinfo\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.614493 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.614335 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-workload-certs\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.614493 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.614356 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-istio-envoy\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.614493 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.614382 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-istio-data\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.614493 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.614452 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/828c54da-1615-4817-9988-d940371838b6-istio-token\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.614493 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.614474 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-credential-socket\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.614783 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.614625 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/828c54da-1615-4817-9988-d940371838b6-istiod-ca-cert\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.614783 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.614675 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frc4m\" (UniqueName: \"kubernetes.io/projected/828c54da-1615-4817-9988-d940371838b6-kube-api-access-frc4m\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.614783 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.614742 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-workload-socket\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.615007 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.614884 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-workload-certs\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.615072 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.615048 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-credential-socket\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.615125 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.615105 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-workload-socket\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.615208 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.615189 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-istio-data\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.615309 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.615293 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/828c54da-1615-4817-9988-d940371838b6-istiod-ca-cert\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.616696 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.616680 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-istio-envoy\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.616825 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.616808 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/828c54da-1615-4817-9988-d940371838b6-istio-podinfo\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.638383 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.638313 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/828c54da-1615-4817-9988-d940371838b6-istio-token\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.638498 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.638421 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frc4m\" (UniqueName: \"kubernetes.io/projected/828c54da-1615-4817-9988-d940371838b6-kube-api-access-frc4m\") pod \"e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.698851 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.698822 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:12.712378 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.712350 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:12.913570 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:45:12.913538 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod828c54da_1615_4817_9988_d940371838b6.slice/crio-4856209fcf9e569ac11d5b46f9fca1e31b5231b2dd506550eb55809c64e9a177 WatchSource:0}: Error finding container 4856209fcf9e569ac11d5b46f9fca1e31b5231b2dd506550eb55809c64e9a177: Status 404 returned error can't find the container with id 4856209fcf9e569ac11d5b46f9fca1e31b5231b2dd506550eb55809c64e9a177 Jul 07 13:45:12.913710 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.913613 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4"] Jul 07 13:45:12.915914 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.915881 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:45:12.916028 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.915947 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:45:12.916028 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:12.915975 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:45:13.423111 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:13.423078 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" event={"ID":"828c54da-1615-4817-9988-d940371838b6","Type":"ContainerStarted","Data":"4840ae3dacf47d423994d8c6942a2c7edb75552601b5abacee16c77264485233"} Jul 07 13:45:13.423303 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:13.423137 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" event={"ID":"828c54da-1615-4817-9988-d940371838b6","Type":"ContainerStarted","Data":"4856209fcf9e569ac11d5b46f9fca1e31b5231b2dd506550eb55809c64e9a177"} Jul 07 13:45:13.632121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:13.632074 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" podStartSLOduration=1.632057265 podStartE2EDuration="1.632057265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:45:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:45:13.598332557 +0000 UTC m=+1871.276893888" watchObservedRunningTime="2026-07-07 13:45:13.632057265 +0000 UTC m=+1871.310618595" Jul 07 13:45:13.699469 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:13.699388 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:13.703689 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:13.703667 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:14.428747 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:14.428718 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:14.429569 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:14.429554 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:16.369350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:16.369313 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:16.852878 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:16.852850 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:17.183387 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:17.183355 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:18.433899 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:18.433868 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:18.661998 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:18.661966 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:20.721991 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:20.721961 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:21.143461 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.143385 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:21.652748 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.652717 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx"] Jul 07 13:45:21.653215 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.653199 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2af35c88-ad2e-49ef-bb98-ace302681ec6" containerName="istio-proxy" Jul 07 13:45:21.653266 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.653217 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2af35c88-ad2e-49ef-bb98-ace302681ec6" containerName="istio-proxy" Jul 07 13:45:21.653307 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.653299 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2af35c88-ad2e-49ef-bb98-ace302681ec6" containerName="istio-proxy" Jul 07 13:45:21.655685 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.655670 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.665548 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.665529 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-9285f1-openshift-default-dockercfg-2dgxj\"" Jul 07 13:45:21.707948 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.707918 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-workload-certs\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.708144 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.707957 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-token\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.708144 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.707982 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkkqz\" (UniqueName: \"kubernetes.io/projected/41a72dac-0e85-45f7-8f49-5e3484aca5c0-kube-api-access-nkkqz\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.708144 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.708000 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-workload-socket\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.708144 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.708033 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-podinfo\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.708144 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.708052 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-credential-socket\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.708144 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.708127 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-envoy\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.708392 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.708174 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-data\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.708392 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.708284 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istiod-ca-cert\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.717820 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.717792 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx"] Jul 07 13:45:21.770192 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.770165 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:21.809225 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.809195 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istiod-ca-cert\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.809393 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.809231 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-workload-certs\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.809393 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.809253 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-token\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.809393 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.809375 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkkqz\" (UniqueName: \"kubernetes.io/projected/41a72dac-0e85-45f7-8f49-5e3484aca5c0-kube-api-access-nkkqz\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.809552 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.809411 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-workload-socket\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.809552 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.809462 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-podinfo\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.809552 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.809497 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-credential-socket\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.809749 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.809652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-envoy\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.809749 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.809680 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-workload-certs\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.809749 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.809725 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-data\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.809902 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.809873 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-credential-socket\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.810079 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.810059 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-data\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.810152 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.810108 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-workload-socket\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.810204 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.810158 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istiod-ca-cert\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.811787 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.811767 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-envoy\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.811945 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.811927 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-podinfo\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.838100 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.838072 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkkqz\" (UniqueName: \"kubernetes.io/projected/41a72dac-0e85-45f7-8f49-5e3484aca5c0-kube-api-access-nkkqz\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.842152 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.842126 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-token\") pod \"e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:21.966249 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:21.966218 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:22.054231 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:22.054206 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:22.142338 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:22.142315 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx"] Jul 07 13:45:22.144382 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:45:22.144356 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a72dac_0e85_45f7_8f49_5e3484aca5c0.slice/crio-6157e36afa6122c67636e6740c427cb392aa8857da2ba69a5fc3087304bceba9 WatchSource:0}: Error finding container 6157e36afa6122c67636e6740c427cb392aa8857da2ba69a5fc3087304bceba9: Status 404 returned error can't find the container with id 6157e36afa6122c67636e6740c427cb392aa8857da2ba69a5fc3087304bceba9 Jul 07 13:45:22.146412 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:22.146380 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:45:22.146494 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:22.146460 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:45:22.146555 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:22.146504 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:45:22.472046 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:22.472006 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" event={"ID":"41a72dac-0e85-45f7-8f49-5e3484aca5c0","Type":"ContainerStarted","Data":"b3aeba1ceaa4a1338667e846de1818be5f98ec1b359b04d75f507afcc0f6fbf4"} Jul 07 13:45:22.472247 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:22.472063 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" event={"ID":"41a72dac-0e85-45f7-8f49-5e3484aca5c0","Type":"ContainerStarted","Data":"6157e36afa6122c67636e6740c427cb392aa8857da2ba69a5fc3087304bceba9"} Jul 07 13:45:22.515511 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:22.515462 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" podStartSLOduration=1.515447659 podStartE2EDuration="1.515447659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:45:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:45:22.511842699 +0000 UTC m=+1880.190404030" watchObservedRunningTime="2026-07-07 13:45:22.515447659 +0000 UTC m=+1880.194008990" Jul 07 13:45:22.972508 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:22.972480 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:23.971802 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:23.971766 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:24.482678 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:24.482648 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:24.483499 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:24.483483 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:25.042702 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:25.042658 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:25.279417 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:25.279380 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:25.541766 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:25.541731 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:26.562847 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:26.562812 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:26.781867 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:26.781832 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:29.320758 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:29.320702 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:30.470760 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:30.470723 2568 patch_prober.go:28] interesting pod/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 13:45:30.471131 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:30.470789 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" podUID="828c54da-1615-4817-9988-d940371838b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 13:45:32.526737 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:32.526702 2568 generic.go:356] "Generic (PLEG): container finished" podID="828c54da-1615-4817-9988-d940371838b6" containerID="4840ae3dacf47d423994d8c6942a2c7edb75552601b5abacee16c77264485233" exitCode=0 Jul 07 13:45:32.527206 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:32.526775 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" event={"ID":"828c54da-1615-4817-9988-d940371838b6","Type":"ContainerDied","Data":"4840ae3dacf47d423994d8c6942a2c7edb75552601b5abacee16c77264485233"} Jul 07 13:45:32.527294 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:32.527278 2568 scope.go:117] "RemoveContainer" containerID="4840ae3dacf47d423994d8c6942a2c7edb75552601b5abacee16c77264485233" Jul 07 13:45:32.528556 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:32.528507 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:45:32.528678 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:32.528614 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:45:32.528678 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:32.528666 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:45:33.538536 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:33.538492 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" event={"ID":"828c54da-1615-4817-9988-d940371838b6","Type":"ContainerStarted","Data":"90725fc2bc5636b3601693b31490f38949204ea3a2158a66f730ee8595c6d1ac"} Jul 07 13:45:33.699199 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:33.699158 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:33.699386 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:33.699230 2568 patch_prober.go:28] interesting pod/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 07 13:45:33.699386 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:33.699274 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" podUID="828c54da-1615-4817-9988-d940371838b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 07 13:45:34.700033 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:34.699993 2568 patch_prober.go:28] interesting pod/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 07 13:45:34.700436 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:34.700055 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" podUID="828c54da-1615-4817-9988-d940371838b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 07 13:45:35.699935 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:35.699896 2568 patch_prober.go:28] interesting pod/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 07 13:45:35.700121 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:35.699956 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" podUID="828c54da-1615-4817-9988-d940371838b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 07 13:45:36.699352 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:36.699310 2568 patch_prober.go:28] interesting pod/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 07 13:45:36.699530 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:36.699395 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" podUID="828c54da-1615-4817-9988-d940371838b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 07 13:45:37.704365 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:37.704327 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:37.704844 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:37.704826 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:37.705596 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:37.705551 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:45:38.573658 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:38.573622 2568 generic.go:356] "Generic (PLEG): container finished" podID="41a72dac-0e85-45f7-8f49-5e3484aca5c0" containerID="b3aeba1ceaa4a1338667e846de1818be5f98ec1b359b04d75f507afcc0f6fbf4" exitCode=0 Jul 07 13:45:38.573871 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:38.573697 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" event={"ID":"41a72dac-0e85-45f7-8f49-5e3484aca5c0","Type":"ContainerDied","Data":"b3aeba1ceaa4a1338667e846de1818be5f98ec1b359b04d75f507afcc0f6fbf4"} Jul 07 13:45:38.574282 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:38.574263 2568 scope.go:117] "RemoveContainer" containerID="b3aeba1ceaa4a1338667e846de1818be5f98ec1b359b04d75f507afcc0f6fbf4" Jul 07 13:45:38.575342 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:38.575316 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:45:38.575400 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:38.575387 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:45:38.575450 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:38.575438 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:45:39.482826 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:39.482789 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:39.581069 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:39.581035 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" event={"ID":"41a72dac-0e85-45f7-8f49-5e3484aca5c0","Type":"ContainerStarted","Data":"25f32b393f413e4af5b35b7bd967d48f35ba559088d969b30bd74bc84e5c0749"} Jul 07 13:45:39.731824 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:39.731776 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:39.855065 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:39.854950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:39.966677 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:39.966631 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:39.967940 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:39.967913 2568 patch_prober.go:28] interesting pod/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 07 13:45:39.968068 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:39.967976 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" podUID="41a72dac-0e85-45f7-8f49-5e3484aca5c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 07 13:45:40.966615 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:40.966562 2568 patch_prober.go:28] interesting pod/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 07 13:45:40.967087 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:40.966643 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" podUID="41a72dac-0e85-45f7-8f49-5e3484aca5c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 07 13:45:41.966840 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:41.966804 2568 patch_prober.go:28] interesting pod/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 07 13:45:41.967324 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:41.966866 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" podUID="41a72dac-0e85-45f7-8f49-5e3484aca5c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 07 13:45:42.967522 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:42.967487 2568 patch_prober.go:28] interesting pod/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 07 13:45:42.967957 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:42.967552 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" podUID="41a72dac-0e85-45f7-8f49-5e3484aca5c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 07 13:45:43.970997 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:43.970966 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:43.971370 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:43.971338 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:43.972103 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:43.972082 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:45:44.621178 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:44.621149 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:47.801024 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:47.800982 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:45:48.139120 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:45:48.139040 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:46:12.840412 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:46:12.840381 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:46:43.762265 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:46:43.762232 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:46:44.666683 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:46:44.666650 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:46:44.839432 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:46:44.839400 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:46:45.242426 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:46:45.242390 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:46:45.349553 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:46:45.349513 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:46:45.458114 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:46:45.458085 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:46:45.672699 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:46:45.672674 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:46:50.311012 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:46:50.310981 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:27.748516 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:27.748488 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:28.055706 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:28.055609 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4"] Jul 07 13:47:28.055895 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:28.055835 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" podUID="828c54da-1615-4817-9988-d940371838b6" containerName="istio-proxy" containerID="cri-o://90725fc2bc5636b3601693b31490f38949204ea3a2158a66f730ee8595c6d1ac" gracePeriod=30 Jul 07 13:47:28.069652 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:28.069621 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:28.171741 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:28.171706 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:28.279317 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:28.279288 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:28.386672 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:28.386594 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:28.476088 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:28.476049 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:28.985897 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:28.985868 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:29.275721 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:29.275633 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:29.407166 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:29.407132 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:29.623246 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:29.623172 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:33.190984 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.190953 2568 generic.go:356] "Generic (PLEG): container finished" podID="828c54da-1615-4817-9988-d940371838b6" containerID="90725fc2bc5636b3601693b31490f38949204ea3a2158a66f730ee8595c6d1ac" exitCode=0 Jul 07 13:47:33.191361 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.191000 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" event={"ID":"828c54da-1615-4817-9988-d940371838b6","Type":"ContainerDied","Data":"90725fc2bc5636b3601693b31490f38949204ea3a2158a66f730ee8595c6d1ac"} Jul 07 13:47:33.191361 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.191069 2568 scope.go:117] "RemoveContainer" containerID="4840ae3dacf47d423994d8c6942a2c7edb75552601b5abacee16c77264485233" Jul 07 13:47:33.313209 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.313183 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:47:33.472671 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.472640 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/828c54da-1615-4817-9988-d940371838b6-istio-token\") pod \"828c54da-1615-4817-9988-d940371838b6\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " Jul 07 13:47:33.472871 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.472698 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-workload-socket\") pod \"828c54da-1615-4817-9988-d940371838b6\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " Jul 07 13:47:33.472871 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.472754 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-istio-data\") pod \"828c54da-1615-4817-9988-d940371838b6\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " Jul 07 13:47:33.472871 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.472779 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-workload-certs\") pod \"828c54da-1615-4817-9988-d940371838b6\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " Jul 07 13:47:33.472871 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.472818 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frc4m\" (UniqueName: \"kubernetes.io/projected/828c54da-1615-4817-9988-d940371838b6-kube-api-access-frc4m\") pod \"828c54da-1615-4817-9988-d940371838b6\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " Jul 07 13:47:33.472871 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.472851 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/828c54da-1615-4817-9988-d940371838b6-istiod-ca-cert\") pod \"828c54da-1615-4817-9988-d940371838b6\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " Jul 07 13:47:33.473142 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.472933 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/828c54da-1615-4817-9988-d940371838b6-istio-podinfo\") pod \"828c54da-1615-4817-9988-d940371838b6\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " Jul 07 13:47:33.473142 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.472968 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-credential-socket\") pod \"828c54da-1615-4817-9988-d940371838b6\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " Jul 07 13:47:33.473142 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.473052 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "828c54da-1615-4817-9988-d940371838b6" (UID: "828c54da-1615-4817-9988-d940371838b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:33.473142 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.473089 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "828c54da-1615-4817-9988-d940371838b6" (UID: "828c54da-1615-4817-9988-d940371838b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:33.473142 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.473110 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-istio-envoy\") pod \"828c54da-1615-4817-9988-d940371838b6\" (UID: \"828c54da-1615-4817-9988-d940371838b6\") " Jul 07 13:47:33.473372 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.473174 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "828c54da-1615-4817-9988-d940371838b6" (UID: "828c54da-1615-4817-9988-d940371838b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:33.473461 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.473419 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/828c54da-1615-4817-9988-d940371838b6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "828c54da-1615-4817-9988-d940371838b6" (UID: "828c54da-1615-4817-9988-d940371838b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:47:33.473689 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.473569 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-credential-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:33.473783 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.473695 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-workload-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:33.473783 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.473718 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-workload-certs\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:33.474130 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.474104 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-istio-data" (OuterVolumeSpecName: "istio-data") pod "828c54da-1615-4817-9988-d940371838b6" (UID: "828c54da-1615-4817-9988-d940371838b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:33.475054 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.475030 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/828c54da-1615-4817-9988-d940371838b6-istio-token" (OuterVolumeSpecName: "istio-token") pod "828c54da-1615-4817-9988-d940371838b6" (UID: "828c54da-1615-4817-9988-d940371838b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:47:33.475054 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.475040 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "828c54da-1615-4817-9988-d940371838b6" (UID: "828c54da-1615-4817-9988-d940371838b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:33.475287 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.475264 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/828c54da-1615-4817-9988-d940371838b6-kube-api-access-frc4m" (OuterVolumeSpecName: "kube-api-access-frc4m") pod "828c54da-1615-4817-9988-d940371838b6" (UID: "828c54da-1615-4817-9988-d940371838b6"). InnerVolumeSpecName "kube-api-access-frc4m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:47:33.475481 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.475462 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/828c54da-1615-4817-9988-d940371838b6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "828c54da-1615-4817-9988-d940371838b6" (UID: "828c54da-1615-4817-9988-d940371838b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:47:33.574987 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.574962 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/828c54da-1615-4817-9988-d940371838b6-istio-podinfo\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:33.574987 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.574984 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-istio-envoy\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:33.575119 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.574994 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/828c54da-1615-4817-9988-d940371838b6-istio-token\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:33.575119 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.575002 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/828c54da-1615-4817-9988-d940371838b6-istio-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:33.575119 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.575010 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frc4m\" (UniqueName: \"kubernetes.io/projected/828c54da-1615-4817-9988-d940371838b6-kube-api-access-frc4m\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:33.575119 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:33.575018 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/828c54da-1615-4817-9988-d940371838b6-istiod-ca-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:34.198284 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:34.198249 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" event={"ID":"828c54da-1615-4817-9988-d940371838b6","Type":"ContainerDied","Data":"4856209fcf9e569ac11d5b46f9fca1e31b5231b2dd506550eb55809c64e9a177"} Jul 07 13:47:34.198284 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:34.198277 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4" Jul 07 13:47:34.198798 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:34.198308 2568 scope.go:117] "RemoveContainer" containerID="90725fc2bc5636b3601693b31490f38949204ea3a2158a66f730ee8595c6d1ac" Jul 07 13:47:34.225132 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:34.225100 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4"] Jul 07 13:47:34.229294 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:34.229270 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-055c02-openshift-default-d4f986759-hhnr4"] Jul 07 13:47:34.971118 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:34.971089 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="828c54da-1615-4817-9988-d940371838b6" path="/var/lib/kubelet/pods/828c54da-1615-4817-9988-d940371838b6/volumes" Jul 07 13:47:35.724722 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:35.724689 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:35.853070 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:35.853039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:35.993221 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:35.993130 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx"] Jul 07 13:47:35.993386 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:35.993345 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" podUID="41a72dac-0e85-45f7-8f49-5e3484aca5c0" containerName="istio-proxy" containerID="cri-o://25f32b393f413e4af5b35b7bd967d48f35ba559088d969b30bd74bc84e5c0749" gracePeriod=30 Jul 07 13:47:36.041628 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:36.041595 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:36.276252 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:36.276165 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:36.608611 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:36.608523 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:36.959160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:36.959133 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk"] Jul 07 13:47:36.959702 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:36.959687 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="828c54da-1615-4817-9988-d940371838b6" containerName="istio-proxy" Jul 07 13:47:36.959761 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:36.959705 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="828c54da-1615-4817-9988-d940371838b6" containerName="istio-proxy" Jul 07 13:47:36.959761 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:36.959726 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="828c54da-1615-4817-9988-d940371838b6" containerName="istio-proxy" Jul 07 13:47:36.959761 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:36.959733 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="828c54da-1615-4817-9988-d940371838b6" containerName="istio-proxy" Jul 07 13:47:36.959869 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:36.959804 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="828c54da-1615-4817-9988-d940371838b6" containerName="istio-proxy" Jul 07 13:47:36.959869 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:36.959811 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="828c54da-1615-4817-9988-d940371838b6" containerName="istio-proxy" Jul 07 13:47:36.962116 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:36.962102 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:36.965923 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:36.965889 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-843413-openshift-default-dockercfg-9c7sc\"" Jul 07 13:47:36.976697 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:36.976675 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk"] Jul 07 13:47:36.991966 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:36.991942 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:37.063159 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.063133 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:37.110002 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.109970 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-istio-data\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.110176 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.110020 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-workload-socket\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.110176 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.110051 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2103686e-8486-408d-a135-cf5708ef0cfa-istiod-ca-cert\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.110176 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.110130 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2103686e-8486-408d-a135-cf5708ef0cfa-istio-podinfo\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.110338 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.110205 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2103686e-8486-408d-a135-cf5708ef0cfa-istio-token\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.110338 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.110237 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-credential-socket\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.110338 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.110284 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-workload-certs\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.110338 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.110307 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-istio-envoy\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.110535 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.110409 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnwx4\" (UniqueName: \"kubernetes.io/projected/2103686e-8486-408d-a135-cf5708ef0cfa-kube-api-access-jnwx4\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.211181 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.211096 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-workload-certs\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.211181 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.211133 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-istio-envoy\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.211398 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.211222 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnwx4\" (UniqueName: \"kubernetes.io/projected/2103686e-8486-408d-a135-cf5708ef0cfa-kube-api-access-jnwx4\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.211398 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.211265 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-istio-data\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.211398 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.211297 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-workload-socket\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.211398 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.211319 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2103686e-8486-408d-a135-cf5708ef0cfa-istiod-ca-cert\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.211622 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.211403 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2103686e-8486-408d-a135-cf5708ef0cfa-istio-podinfo\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.211622 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.211448 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2103686e-8486-408d-a135-cf5708ef0cfa-istio-token\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.211622 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.211477 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-credential-socket\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.211777 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.211701 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-istio-data\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.212221 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.212193 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2103686e-8486-408d-a135-cf5708ef0cfa-istiod-ca-cert\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.212392 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.212367 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-workload-socket\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.212528 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.212411 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-workload-certs\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.212528 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.212462 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-credential-socket\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.213782 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.213761 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2103686e-8486-408d-a135-cf5708ef0cfa-istio-podinfo\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.213887 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.213856 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-istio-envoy\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.226605 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.222246 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2103686e-8486-408d-a135-cf5708ef0cfa-istio-token\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.226605 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.222512 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnwx4\" (UniqueName: \"kubernetes.io/projected/2103686e-8486-408d-a135-cf5708ef0cfa-kube-api-access-jnwx4\") pod \"e2e-api-a-843413-openshift-default-f77fd67d5-td6xk\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.273803 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.273764 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:37.417198 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.417168 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk"] Jul 07 13:47:37.420338 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:47:37.420309 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2103686e_8486_408d_a135_cf5708ef0cfa.slice/crio-324f388803c697b45b4d4816b1afd8876e3636f75f6c41bff257894ab68b261a WatchSource:0}: Error finding container 324f388803c697b45b4d4816b1afd8876e3636f75f6c41bff257894ab68b261a: Status 404 returned error can't find the container with id 324f388803c697b45b4d4816b1afd8876e3636f75f6c41bff257894ab68b261a Jul 07 13:47:37.422683 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.422641 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:47:37.422795 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.422711 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:47:37.422795 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:37.422746 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:47:38.223545 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:38.223509 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" event={"ID":"2103686e-8486-408d-a135-cf5708ef0cfa","Type":"ContainerStarted","Data":"1646f26cd2f66cd25eb719737ca53c4e75c36eff7311b390f513e30b81bd1018"} Jul 07 13:47:38.224007 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:38.223596 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" event={"ID":"2103686e-8486-408d-a135-cf5708ef0cfa","Type":"ContainerStarted","Data":"324f388803c697b45b4d4816b1afd8876e3636f75f6c41bff257894ab68b261a"} Jul 07 13:47:38.260847 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:38.260798 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" podStartSLOduration=2.26077808 podStartE2EDuration="2.26077808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:47:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:47:38.257090528 +0000 UTC m=+2015.935651858" watchObservedRunningTime="2026-07-07 13:47:38.26077808 +0000 UTC m=+2015.939339411" Jul 07 13:47:38.274501 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:38.274475 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:38.279293 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:38.279272 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:39.228596 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:39.228557 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:39.229635 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:39.229614 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:40.289975 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:40.289943 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:40.508081 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:40.508050 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:40.765511 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:40.765483 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:40.912885 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:40.912853 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:41.241678 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.241651 2568 generic.go:356] "Generic (PLEG): container finished" podID="41a72dac-0e85-45f7-8f49-5e3484aca5c0" containerID="25f32b393f413e4af5b35b7bd967d48f35ba559088d969b30bd74bc84e5c0749" exitCode=0 Jul 07 13:47:41.241802 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.241726 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" event={"ID":"41a72dac-0e85-45f7-8f49-5e3484aca5c0","Type":"ContainerDied","Data":"25f32b393f413e4af5b35b7bd967d48f35ba559088d969b30bd74bc84e5c0749"} Jul 07 13:47:41.241802 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.241783 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" event={"ID":"41a72dac-0e85-45f7-8f49-5e3484aca5c0","Type":"ContainerDied","Data":"6157e36afa6122c67636e6740c427cb392aa8857da2ba69a5fc3087304bceba9"} Jul 07 13:47:41.241876 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.241805 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6157e36afa6122c67636e6740c427cb392aa8857da2ba69a5fc3087304bceba9" Jul 07 13:47:41.241876 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.241821 2568 scope.go:117] "RemoveContainer" containerID="b3aeba1ceaa4a1338667e846de1818be5f98ec1b359b04d75f507afcc0f6fbf4" Jul 07 13:47:41.256764 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.256748 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:47:41.347497 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.347469 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-workload-certs\") pod \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " Jul 07 13:47:41.347497 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.347502 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkkqz\" (UniqueName: \"kubernetes.io/projected/41a72dac-0e85-45f7-8f49-5e3484aca5c0-kube-api-access-nkkqz\") pod \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " Jul 07 13:47:41.347997 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.347544 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-token\") pod \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " Jul 07 13:47:41.347997 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.347605 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-envoy\") pod \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " Jul 07 13:47:41.347997 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.347637 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-data\") pod \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " Jul 07 13:47:41.347997 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.347679 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-workload-socket\") pod \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " Jul 07 13:47:41.347997 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.347782 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "41a72dac-0e85-45f7-8f49-5e3484aca5c0" (UID: "41a72dac-0e85-45f7-8f49-5e3484aca5c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:41.347997 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.347824 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-podinfo\") pod \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " Jul 07 13:47:41.347997 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.347900 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-credential-socket\") pod \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " Jul 07 13:47:41.348355 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.348053 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "41a72dac-0e85-45f7-8f49-5e3484aca5c0" (UID: "41a72dac-0e85-45f7-8f49-5e3484aca5c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:41.348355 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.348337 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "41a72dac-0e85-45f7-8f49-5e3484aca5c0" (UID: "41a72dac-0e85-45f7-8f49-5e3484aca5c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:41.348550 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.348518 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istiod-ca-cert\") pod \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\" (UID: \"41a72dac-0e85-45f7-8f49-5e3484aca5c0\") " Jul 07 13:47:41.348657 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.348628 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "41a72dac-0e85-45f7-8f49-5e3484aca5c0" (UID: "41a72dac-0e85-45f7-8f49-5e3484aca5c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:41.348867 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.348841 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "41a72dac-0e85-45f7-8f49-5e3484aca5c0" (UID: "41a72dac-0e85-45f7-8f49-5e3484aca5c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:47:41.348974 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.348929 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:41.348974 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.348946 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-workload-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:41.348974 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.348956 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-credential-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:41.348974 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.348964 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istiod-ca-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:41.348974 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.348972 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-workload-certs\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:41.349937 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.349915 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a72dac-0e85-45f7-8f49-5e3484aca5c0-kube-api-access-nkkqz" (OuterVolumeSpecName: "kube-api-access-nkkqz") pod "41a72dac-0e85-45f7-8f49-5e3484aca5c0" (UID: "41a72dac-0e85-45f7-8f49-5e3484aca5c0"). InnerVolumeSpecName "kube-api-access-nkkqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:47:41.350024 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.350002 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "41a72dac-0e85-45f7-8f49-5e3484aca5c0" (UID: "41a72dac-0e85-45f7-8f49-5e3484aca5c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:41.350253 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.350231 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "41a72dac-0e85-45f7-8f49-5e3484aca5c0" (UID: "41a72dac-0e85-45f7-8f49-5e3484aca5c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:47:41.350253 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.350242 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "41a72dac-0e85-45f7-8f49-5e3484aca5c0" (UID: "41a72dac-0e85-45f7-8f49-5e3484aca5c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:47:41.413240 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.413175 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:41.449764 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.449737 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-podinfo\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:41.449764 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.449758 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkkqz\" (UniqueName: \"kubernetes.io/projected/41a72dac-0e85-45f7-8f49-5e3484aca5c0-kube-api-access-nkkqz\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:41.449764 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.449767 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-token\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:41.449952 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.449775 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a72dac-0e85-45f7-8f49-5e3484aca5c0-istio-envoy\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:47:41.553114 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:41.553039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:42.247983 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:42.247956 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx" Jul 07 13:47:42.322104 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:42.322074 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx"] Jul 07 13:47:42.324511 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:42.324482 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-9285f1-openshift-default-6dcc8c76b9-pdtbx"] Jul 07 13:47:42.970918 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:42.970885 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41a72dac-0e85-45f7-8f49-5e3484aca5c0" path="/var/lib/kubelet/pods/41a72dac-0e85-45f7-8f49-5e3484aca5c0/volumes" Jul 07 13:47:44.787701 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:44.787671 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:45.292437 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.292409 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:45.639953 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.639871 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf"] Jul 07 13:47:45.640335 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.640323 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41a72dac-0e85-45f7-8f49-5e3484aca5c0" containerName="istio-proxy" Jul 07 13:47:45.640389 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.640337 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="41a72dac-0e85-45f7-8f49-5e3484aca5c0" containerName="istio-proxy" Jul 07 13:47:45.640389 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.640353 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41a72dac-0e85-45f7-8f49-5e3484aca5c0" containerName="istio-proxy" Jul 07 13:47:45.640389 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.640358 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="41a72dac-0e85-45f7-8f49-5e3484aca5c0" containerName="istio-proxy" Jul 07 13:47:45.640485 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.640426 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="41a72dac-0e85-45f7-8f49-5e3484aca5c0" containerName="istio-proxy" Jul 07 13:47:45.640485 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.640437 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="41a72dac-0e85-45f7-8f49-5e3484aca5c0" containerName="istio-proxy" Jul 07 13:47:45.643556 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.643535 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.646356 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.646331 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-fe52df-openshift-default-dockercfg-lcgzc\"" Jul 07 13:47:45.661753 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.661732 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf"] Jul 07 13:47:45.688643 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.688615 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-envoy\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.688759 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.688667 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-data\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.688759 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.688743 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-workload-socket\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.688844 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.688823 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cc33c45-f020-457e-b62b-f390cf38ddb7-istiod-ca-cert\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.688906 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.688889 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-workload-certs\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.688970 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.688948 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llvzs\" (UniqueName: \"kubernetes.io/projected/3cc33c45-f020-457e-b62b-f390cf38ddb7-kube-api-access-llvzs\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.689078 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.689013 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-podinfo\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.689078 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.689041 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-token\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.689171 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.689155 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-credential-socket\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.722027 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.722004 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:45.790102 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.790065 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-credential-socket\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.790541 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.790122 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-envoy\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.790541 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.790166 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-data\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.790541 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.790211 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-workload-socket\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.790541 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.790247 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cc33c45-f020-457e-b62b-f390cf38ddb7-istiod-ca-cert\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.790541 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.790298 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-workload-certs\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.790541 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.790330 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llvzs\" (UniqueName: \"kubernetes.io/projected/3cc33c45-f020-457e-b62b-f390cf38ddb7-kube-api-access-llvzs\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.790541 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.790378 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-podinfo\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.790541 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.790405 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-token\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.790541 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.790531 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-credential-socket\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.790986 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.790551 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-data\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.790986 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.790680 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-workload-socket\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.790986 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.790819 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-workload-certs\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.790986 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.790968 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cc33c45-f020-457e-b62b-f390cf38ddb7-istiod-ca-cert\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.792703 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.792680 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-envoy\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.792904 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.792886 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-podinfo\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.801777 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.801748 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-token\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.802117 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.802099 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llvzs\" (UniqueName: \"kubernetes.io/projected/3cc33c45-f020-457e-b62b-f390cf38ddb7-kube-api-access-llvzs\") pod \"e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:45.829260 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.829196 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:45.954271 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:45.954239 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:46.105880 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:46.105854 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf"] Jul 07 13:47:46.106786 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:47:46.106754 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cc33c45_f020_457e_b62b_f390cf38ddb7.slice/crio-3672dae6698752626f9bb2daa67d5ec0ff6f4ab234baff76f0deae7c8b6ef984 WatchSource:0}: Error finding container 3672dae6698752626f9bb2daa67d5ec0ff6f4ab234baff76f0deae7c8b6ef984: Status 404 returned error can't find the container with id 3672dae6698752626f9bb2daa67d5ec0ff6f4ab234baff76f0deae7c8b6ef984 Jul 07 13:47:46.109127 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:46.109090 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:47:46.109232 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:46.109157 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:47:46.109232 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:46.109185 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:47:46.271351 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:46.271254 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" event={"ID":"3cc33c45-f020-457e-b62b-f390cf38ddb7","Type":"ContainerStarted","Data":"b288a2252169d441a7c95a3ac7fd129ca7393b08644a0f04f6ab3e0707d851f6"} Jul 07 13:47:46.271351 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:46.271326 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" event={"ID":"3cc33c45-f020-457e-b62b-f390cf38ddb7","Type":"ContainerStarted","Data":"3672dae6698752626f9bb2daa67d5ec0ff6f4ab234baff76f0deae7c8b6ef984"} Jul 07 13:47:46.305827 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:46.305753 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" podStartSLOduration=1.3057293300000001 podStartE2EDuration="1.30572933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:47:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:47:46.30260399 +0000 UTC m=+2023.981165316" watchObservedRunningTime="2026-07-07 13:47:46.30572933 +0000 UTC m=+2023.984290662" Jul 07 13:47:46.955215 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:46.955172 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:47.960453 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:47.960415 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:48.282566 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:48.282467 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:48.283648 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:48.283620 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:47:48.456461 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:48.456427 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:50.417951 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:50.417888 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:52.807382 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:52.807353 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:47:53.311110 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:53.311029 2568 generic.go:356] "Generic (PLEG): container finished" podID="2103686e-8486-408d-a135-cf5708ef0cfa" containerID="1646f26cd2f66cd25eb719737ca53c4e75c36eff7311b390f513e30b81bd1018" exitCode=0 Jul 07 13:47:53.311110 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:53.311099 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" event={"ID":"2103686e-8486-408d-a135-cf5708ef0cfa","Type":"ContainerDied","Data":"1646f26cd2f66cd25eb719737ca53c4e75c36eff7311b390f513e30b81bd1018"} Jul 07 13:47:53.311522 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:53.311506 2568 scope.go:117] "RemoveContainer" containerID="1646f26cd2f66cd25eb719737ca53c4e75c36eff7311b390f513e30b81bd1018" Jul 07 13:47:53.312568 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:53.312538 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:47:53.312729 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:53.312639 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:47:53.312729 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:53.312687 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:47:54.228987 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:54.228897 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:54.318015 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:54.317980 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" event={"ID":"2103686e-8486-408d-a135-cf5708ef0cfa","Type":"ContainerStarted","Data":"c1179f9005662b7528441c632acf48882b09dbee0e533124c1a256e7437b7077"} Jul 07 13:47:55.274969 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:55.274933 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:55.276262 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:55.276234 2568 patch_prober.go:28] interesting pod/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 07 13:47:55.276382 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:55.276296 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" podUID="2103686e-8486-408d-a135-cf5708ef0cfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 07 13:47:56.275174 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:56.275135 2568 patch_prober.go:28] interesting pod/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 07 13:47:56.275554 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:56.275208 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" podUID="2103686e-8486-408d-a135-cf5708ef0cfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 07 13:47:57.276016 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:57.275975 2568 patch_prober.go:28] interesting pod/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 07 13:47:57.276653 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:57.276615 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" podUID="2103686e-8486-408d-a135-cf5708ef0cfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 07 13:47:58.278151 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:58.278119 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:58.278543 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:58.278386 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:47:58.279104 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:47:58.279089 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:48:01.084194 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:01.084158 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:01.688446 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:01.688411 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:01.952224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:01.952139 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:02.067357 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:02.067327 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:02.447220 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:02.447180 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:03.862732 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:03.862554 2568 patch_prober.go:28] interesting pod/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": read tcp 10.134.0.2:59322->10.134.0.52:15021: read: connection reset by peer" start-of-body= Jul 07 13:48:03.862732 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:03.862655 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" podUID="3cc33c45-f020-457e-b62b-f390cf38ddb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": read tcp 10.134.0.2:59322->10.134.0.52:15021: read: connection reset by peer" Jul 07 13:48:04.385098 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:04.385066 2568 generic.go:356] "Generic (PLEG): container finished" podID="3cc33c45-f020-457e-b62b-f390cf38ddb7" containerID="b288a2252169d441a7c95a3ac7fd129ca7393b08644a0f04f6ab3e0707d851f6" exitCode=0 Jul 07 13:48:04.385282 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:04.385154 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" event={"ID":"3cc33c45-f020-457e-b62b-f390cf38ddb7","Type":"ContainerDied","Data":"b288a2252169d441a7c95a3ac7fd129ca7393b08644a0f04f6ab3e0707d851f6"} Jul 07 13:48:04.385614 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:04.385601 2568 scope.go:117] "RemoveContainer" containerID="b288a2252169d441a7c95a3ac7fd129ca7393b08644a0f04f6ab3e0707d851f6" Jul 07 13:48:04.386718 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:04.386691 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:04.386783 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:04.386768 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:04.386817 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:04.386799 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:05.392835 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:05.392802 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" event={"ID":"3cc33c45-f020-457e-b62b-f390cf38ddb7","Type":"ContainerStarted","Data":"8cc1174d68ab774b8e386b1f68c13771726bffc2d7f7ad40cd104b9a789bff7f"} Jul 07 13:48:05.955304 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:05.955257 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:48:05.955541 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:05.955514 2568 patch_prober.go:28] interesting pod/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 07 13:48:05.955645 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:05.955597 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" podUID="3cc33c45-f020-457e-b62b-f390cf38ddb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 07 13:48:06.955446 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:06.955412 2568 patch_prober.go:28] interesting pod/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 07 13:48:06.955889 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:06.955532 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" podUID="3cc33c45-f020-457e-b62b-f390cf38ddb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 07 13:48:07.955448 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:07.955410 2568 patch_prober.go:28] interesting pod/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 07 13:48:07.955872 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:07.955469 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" podUID="3cc33c45-f020-457e-b62b-f390cf38ddb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 07 13:48:08.144040 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:08.144000 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:08.253982 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:08.253871 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:08.455600 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:08.454834 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk"] Jul 07 13:48:08.455600 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:08.455111 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" podUID="2103686e-8486-408d-a135-cf5708ef0cfa" containerName="istio-proxy" containerID="cri-o://c1179f9005662b7528441c632acf48882b09dbee0e533124c1a256e7437b7077" gracePeriod=30 Jul 07 13:48:08.506063 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:08.505964 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:08.701345 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:08.701318 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:08.955194 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:08.955161 2568 patch_prober.go:28] interesting pod/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 07 13:48:08.955374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:08.955247 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" podUID="3cc33c45-f020-457e-b62b-f390cf38ddb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 07 13:48:09.001810 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:09.001780 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:09.397896 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:09.397781 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:09.666297 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:09.666265 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:09.814095 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:09.814017 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:09.964653 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:09.964524 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:48:09.966104 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:09.966083 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:48:09.970378 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:09.970362 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:48:09.998690 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:09.998661 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:13.280218 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.280187 2568 patch_prober.go:28] interesting pod/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 13:48:13.280595 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.280249 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" podUID="2103686e-8486-408d-a135-cf5708ef0cfa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 13:48:13.718167 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.718134 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:48:13.814392 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.814359 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2103686e-8486-408d-a135-cf5708ef0cfa-istio-podinfo\") pod \"2103686e-8486-408d-a135-cf5708ef0cfa\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " Jul 07 13:48:13.814611 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.814414 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2103686e-8486-408d-a135-cf5708ef0cfa-istio-token\") pod \"2103686e-8486-408d-a135-cf5708ef0cfa\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " Jul 07 13:48:13.814611 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.814434 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-credential-socket\") pod \"2103686e-8486-408d-a135-cf5708ef0cfa\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " Jul 07 13:48:13.814611 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.814465 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2103686e-8486-408d-a135-cf5708ef0cfa-istiod-ca-cert\") pod \"2103686e-8486-408d-a135-cf5708ef0cfa\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " Jul 07 13:48:13.814611 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.814499 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnwx4\" (UniqueName: \"kubernetes.io/projected/2103686e-8486-408d-a135-cf5708ef0cfa-kube-api-access-jnwx4\") pod \"2103686e-8486-408d-a135-cf5708ef0cfa\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " Jul 07 13:48:13.814611 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.814545 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-workload-socket\") pod \"2103686e-8486-408d-a135-cf5708ef0cfa\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " Jul 07 13:48:13.814886 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.814634 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-workload-certs\") pod \"2103686e-8486-408d-a135-cf5708ef0cfa\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " Jul 07 13:48:13.814886 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.814682 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-istio-envoy\") pod \"2103686e-8486-408d-a135-cf5708ef0cfa\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " Jul 07 13:48:13.814886 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.814708 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-istio-data\") pod \"2103686e-8486-408d-a135-cf5708ef0cfa\" (UID: \"2103686e-8486-408d-a135-cf5708ef0cfa\") " Jul 07 13:48:13.814886 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.814849 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2103686e-8486-408d-a135-cf5708ef0cfa" (UID: "2103686e-8486-408d-a135-cf5708ef0cfa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:48:13.815130 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.815007 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2103686e-8486-408d-a135-cf5708ef0cfa" (UID: "2103686e-8486-408d-a135-cf5708ef0cfa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:48:13.815130 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.815029 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2103686e-8486-408d-a135-cf5708ef0cfa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2103686e-8486-408d-a135-cf5708ef0cfa" (UID: "2103686e-8486-408d-a135-cf5708ef0cfa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:48:13.815238 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.815139 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2103686e-8486-408d-a135-cf5708ef0cfa" (UID: "2103686e-8486-408d-a135-cf5708ef0cfa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:48:13.815238 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.815172 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-credential-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:13.815701 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.815678 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-istio-data" (OuterVolumeSpecName: "istio-data") pod "2103686e-8486-408d-a135-cf5708ef0cfa" (UID: "2103686e-8486-408d-a135-cf5708ef0cfa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:48:13.816804 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.816771 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2103686e-8486-408d-a135-cf5708ef0cfa-kube-api-access-jnwx4" (OuterVolumeSpecName: "kube-api-access-jnwx4") pod "2103686e-8486-408d-a135-cf5708ef0cfa" (UID: "2103686e-8486-408d-a135-cf5708ef0cfa"). InnerVolumeSpecName "kube-api-access-jnwx4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:48:13.816891 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.816856 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2103686e-8486-408d-a135-cf5708ef0cfa" (UID: "2103686e-8486-408d-a135-cf5708ef0cfa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:48:13.816993 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.816976 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2103686e-8486-408d-a135-cf5708ef0cfa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2103686e-8486-408d-a135-cf5708ef0cfa" (UID: "2103686e-8486-408d-a135-cf5708ef0cfa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:48:13.817035 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.816982 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2103686e-8486-408d-a135-cf5708ef0cfa-istio-token" (OuterVolumeSpecName: "istio-token") pod "2103686e-8486-408d-a135-cf5708ef0cfa" (UID: "2103686e-8486-408d-a135-cf5708ef0cfa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:48:13.915892 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.915857 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-workload-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:13.915892 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.915884 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-workload-certs\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:13.915892 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.915892 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-istio-envoy\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:13.915892 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.915899 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2103686e-8486-408d-a135-cf5708ef0cfa-istio-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:13.916146 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.915907 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2103686e-8486-408d-a135-cf5708ef0cfa-istio-podinfo\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:13.916146 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.915916 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2103686e-8486-408d-a135-cf5708ef0cfa-istio-token\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:13.916146 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.915924 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2103686e-8486-408d-a135-cf5708ef0cfa-istiod-ca-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:13.916146 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:13.915932 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnwx4\" (UniqueName: \"kubernetes.io/projected/2103686e-8486-408d-a135-cf5708ef0cfa-kube-api-access-jnwx4\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:14.443813 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:14.443770 2568 generic.go:356] "Generic (PLEG): container finished" podID="2103686e-8486-408d-a135-cf5708ef0cfa" containerID="c1179f9005662b7528441c632acf48882b09dbee0e533124c1a256e7437b7077" exitCode=0 Jul 07 13:48:14.443813 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:14.443827 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" Jul 07 13:48:14.444279 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:14.443857 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" event={"ID":"2103686e-8486-408d-a135-cf5708ef0cfa","Type":"ContainerDied","Data":"c1179f9005662b7528441c632acf48882b09dbee0e533124c1a256e7437b7077"} Jul 07 13:48:14.444279 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:14.443910 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk" event={"ID":"2103686e-8486-408d-a135-cf5708ef0cfa","Type":"ContainerDied","Data":"324f388803c697b45b4d4816b1afd8876e3636f75f6c41bff257894ab68b261a"} Jul 07 13:48:14.444279 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:14.443931 2568 scope.go:117] "RemoveContainer" containerID="c1179f9005662b7528441c632acf48882b09dbee0e533124c1a256e7437b7077" Jul 07 13:48:14.453020 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:14.453004 2568 scope.go:117] "RemoveContainer" containerID="1646f26cd2f66cd25eb719737ca53c4e75c36eff7311b390f513e30b81bd1018" Jul 07 13:48:14.460931 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:14.460916 2568 scope.go:117] "RemoveContainer" containerID="c1179f9005662b7528441c632acf48882b09dbee0e533124c1a256e7437b7077" Jul 07 13:48:14.461187 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:48:14.461168 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1179f9005662b7528441c632acf48882b09dbee0e533124c1a256e7437b7077\": container with ID starting with c1179f9005662b7528441c632acf48882b09dbee0e533124c1a256e7437b7077 not found: ID does not exist" containerID="c1179f9005662b7528441c632acf48882b09dbee0e533124c1a256e7437b7077" Jul 07 13:48:14.461233 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:14.461196 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1179f9005662b7528441c632acf48882b09dbee0e533124c1a256e7437b7077"} err="failed to get container status \"c1179f9005662b7528441c632acf48882b09dbee0e533124c1a256e7437b7077\": rpc error: code = NotFound desc = could not find container \"c1179f9005662b7528441c632acf48882b09dbee0e533124c1a256e7437b7077\": container with ID starting with c1179f9005662b7528441c632acf48882b09dbee0e533124c1a256e7437b7077 not found: ID does not exist" Jul 07 13:48:14.461233 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:14.461214 2568 scope.go:117] "RemoveContainer" containerID="1646f26cd2f66cd25eb719737ca53c4e75c36eff7311b390f513e30b81bd1018" Jul 07 13:48:14.461426 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:48:14.461410 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1646f26cd2f66cd25eb719737ca53c4e75c36eff7311b390f513e30b81bd1018\": container with ID starting with 1646f26cd2f66cd25eb719737ca53c4e75c36eff7311b390f513e30b81bd1018 not found: ID does not exist" containerID="1646f26cd2f66cd25eb719737ca53c4e75c36eff7311b390f513e30b81bd1018" Jul 07 13:48:14.461478 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:14.461436 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1646f26cd2f66cd25eb719737ca53c4e75c36eff7311b390f513e30b81bd1018"} err="failed to get container status \"1646f26cd2f66cd25eb719737ca53c4e75c36eff7311b390f513e30b81bd1018\": rpc error: code = NotFound desc = could not find container \"1646f26cd2f66cd25eb719737ca53c4e75c36eff7311b390f513e30b81bd1018\": container with ID starting with 1646f26cd2f66cd25eb719737ca53c4e75c36eff7311b390f513e30b81bd1018 not found: ID does not exist" Jul 07 13:48:14.468160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:14.468138 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk"] Jul 07 13:48:14.473794 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:14.473774 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-843413-openshift-default-f77fd67d5-td6xk"] Jul 07 13:48:14.971889 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:14.971855 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2103686e-8486-408d-a135-cf5708ef0cfa" path="/var/lib/kubelet/pods/2103686e-8486-408d-a135-cf5708ef0cfa/volumes" Jul 07 13:48:16.080400 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:16.080367 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:16.182672 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:16.182635 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:16.374987 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:16.374909 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf"] Jul 07 13:48:16.375137 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:16.375114 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" podUID="3cc33c45-f020-457e-b62b-f390cf38ddb7" containerName="istio-proxy" containerID="cri-o://8cc1174d68ab774b8e386b1f68c13771726bffc2d7f7ad40cd104b9a789bff7f" gracePeriod=30 Jul 07 13:48:16.390816 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:16.390769 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:16.621094 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:16.621044 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:16.983999 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:16.983964 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:17.315814 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.315725 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4"] Jul 07 13:48:17.316225 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.316211 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2103686e-8486-408d-a135-cf5708ef0cfa" containerName="istio-proxy" Jul 07 13:48:17.316272 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.316227 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2103686e-8486-408d-a135-cf5708ef0cfa" containerName="istio-proxy" Jul 07 13:48:17.316272 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.316243 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2103686e-8486-408d-a135-cf5708ef0cfa" containerName="istio-proxy" Jul 07 13:48:17.316272 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.316249 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2103686e-8486-408d-a135-cf5708ef0cfa" containerName="istio-proxy" Jul 07 13:48:17.316377 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.316314 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2103686e-8486-408d-a135-cf5708ef0cfa" containerName="istio-proxy" Jul 07 13:48:17.319756 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.319727 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.322847 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.322822 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-db91fe-openshift-default-dockercfg-lhvmq\"" Jul 07 13:48:17.334405 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.334379 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4"] Jul 07 13:48:17.349218 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.349196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-data\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.349332 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.349228 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-workload-socket\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.349332 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.349249 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-credential-socket\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.349332 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.349267 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-workload-certs\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.349496 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.349474 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-envoy\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.349551 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.349513 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f80cafa-73f0-4027-b8c7-2a9081af5303-istiod-ca-cert\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.349629 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.349612 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-token\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.349688 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.349673 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-podinfo\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.349731 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.349715 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kpb6\" (UniqueName: \"kubernetes.io/projected/9f80cafa-73f0-4027-b8c7-2a9081af5303-kube-api-access-4kpb6\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.357416 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.357397 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:17.451057 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.451020 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kpb6\" (UniqueName: \"kubernetes.io/projected/9f80cafa-73f0-4027-b8c7-2a9081af5303-kube-api-access-4kpb6\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.451057 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.451060 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-data\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.451304 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.451081 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-workload-socket\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.451304 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.451100 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-credential-socket\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.451304 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.451119 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-workload-certs\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.451304 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.451238 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-envoy\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.451304 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.451261 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f80cafa-73f0-4027-b8c7-2a9081af5303-istiod-ca-cert\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.451304 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.451303 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-token\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.451627 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.451356 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-podinfo\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.451627 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.451490 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-workload-socket\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.451627 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.451557 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-credential-socket\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.451627 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.451588 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-data\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.451841 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.451714 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-workload-certs\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.452063 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.452043 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f80cafa-73f0-4027-b8c7-2a9081af5303-istiod-ca-cert\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.453569 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.453550 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-envoy\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.453942 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.453922 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-podinfo\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.462232 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.462207 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kpb6\" (UniqueName: \"kubernetes.io/projected/9f80cafa-73f0-4027-b8c7-2a9081af5303-kube-api-access-4kpb6\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.465824 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.465801 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-token\") pod \"e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.556325 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.556262 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:17.631270 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.631175 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:17.791744 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.791721 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4"] Jul 07 13:48:17.794010 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:48:17.793971 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f80cafa_73f0_4027_b8c7_2a9081af5303.slice/crio-ba572a2276752fbf0ba88e97ce265b02b461463725156cbeaff7445198bf1ac2 WatchSource:0}: Error finding container ba572a2276752fbf0ba88e97ce265b02b461463725156cbeaff7445198bf1ac2: Status 404 returned error can't find the container with id ba572a2276752fbf0ba88e97ce265b02b461463725156cbeaff7445198bf1ac2 Jul 07 13:48:17.796140 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.796106 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:17.796216 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.796172 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:17.796216 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:17.796206 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:18.467374 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:18.467341 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" event={"ID":"9f80cafa-73f0-4027-b8c7-2a9081af5303","Type":"ContainerStarted","Data":"21d47a9b6542dbd0f4a13bb86d57195f5fca33104997c632bf84b119dc0cac01"} Jul 07 13:48:18.467796 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:18.467391 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" event={"ID":"9f80cafa-73f0-4027-b8c7-2a9081af5303","Type":"ContainerStarted","Data":"ba572a2276752fbf0ba88e97ce265b02b461463725156cbeaff7445198bf1ac2"} Jul 07 13:48:18.510377 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:18.510320 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" podStartSLOduration=1.510303977 podStartE2EDuration="1.510303977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:48:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:48:18.509377691 +0000 UTC m=+2056.187939021" watchObservedRunningTime="2026-07-07 13:48:18.510303977 +0000 UTC m=+2056.188865344" Jul 07 13:48:18.631673 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:18.631642 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:18.636864 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:18.636837 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:19.473203 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:19.473167 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:19.474056 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:19.474038 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:20.461244 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:20.461208 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:20.580941 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:20.580885 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:20.665037 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:20.664989 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:20.845795 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:20.845669 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:21.380431 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.380401 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:21.481224 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.481190 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:21.641245 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.641192 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:48:21.702550 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.702520 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llvzs\" (UniqueName: \"kubernetes.io/projected/3cc33c45-f020-457e-b62b-f390cf38ddb7-kube-api-access-llvzs\") pod \"3cc33c45-f020-457e-b62b-f390cf38ddb7\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " Jul 07 13:48:21.702728 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.702567 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-credential-socket\") pod \"3cc33c45-f020-457e-b62b-f390cf38ddb7\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " Jul 07 13:48:21.702728 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.702629 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-workload-socket\") pod \"3cc33c45-f020-457e-b62b-f390cf38ddb7\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " Jul 07 13:48:21.702840 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.702733 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-envoy\") pod \"3cc33c45-f020-457e-b62b-f390cf38ddb7\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " Jul 07 13:48:21.702840 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.702771 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-workload-certs\") pod \"3cc33c45-f020-457e-b62b-f390cf38ddb7\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " Jul 07 13:48:21.702840 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.702802 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-data\") pod \"3cc33c45-f020-457e-b62b-f390cf38ddb7\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " Jul 07 13:48:21.702990 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.702838 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-podinfo\") pod \"3cc33c45-f020-457e-b62b-f390cf38ddb7\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " Jul 07 13:48:21.702990 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.702854 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3cc33c45-f020-457e-b62b-f390cf38ddb7" (UID: "3cc33c45-f020-457e-b62b-f390cf38ddb7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:48:21.702990 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.702869 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-token\") pod \"3cc33c45-f020-457e-b62b-f390cf38ddb7\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " Jul 07 13:48:21.703148 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.703001 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cc33c45-f020-457e-b62b-f390cf38ddb7-istiod-ca-cert\") pod \"3cc33c45-f020-457e-b62b-f390cf38ddb7\" (UID: \"3cc33c45-f020-457e-b62b-f390cf38ddb7\") " Jul 07 13:48:21.703148 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.703039 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3cc33c45-f020-457e-b62b-f390cf38ddb7" (UID: "3cc33c45-f020-457e-b62b-f390cf38ddb7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:48:21.703148 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.703091 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3cc33c45-f020-457e-b62b-f390cf38ddb7" (UID: "3cc33c45-f020-457e-b62b-f390cf38ddb7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:48:21.703345 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.703324 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cc33c45-f020-457e-b62b-f390cf38ddb7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3cc33c45-f020-457e-b62b-f390cf38ddb7" (UID: "3cc33c45-f020-457e-b62b-f390cf38ddb7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:48:21.703643 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.703507 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-workload-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:21.703643 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.703531 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-workload-certs\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:21.704001 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.703549 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cc33c45-f020-457e-b62b-f390cf38ddb7-istiod-ca-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:21.704001 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.703667 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-credential-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:21.704001 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.703963 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-data" (OuterVolumeSpecName: "istio-data") pod "3cc33c45-f020-457e-b62b-f390cf38ddb7" (UID: "3cc33c45-f020-457e-b62b-f390cf38ddb7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:48:21.705273 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.705236 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3cc33c45-f020-457e-b62b-f390cf38ddb7" (UID: "3cc33c45-f020-457e-b62b-f390cf38ddb7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:48:21.705394 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.705361 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-token" (OuterVolumeSpecName: "istio-token") pod "3cc33c45-f020-457e-b62b-f390cf38ddb7" (UID: "3cc33c45-f020-457e-b62b-f390cf38ddb7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:48:21.705459 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.705388 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cc33c45-f020-457e-b62b-f390cf38ddb7-kube-api-access-llvzs" (OuterVolumeSpecName: "kube-api-access-llvzs") pod "3cc33c45-f020-457e-b62b-f390cf38ddb7" (UID: "3cc33c45-f020-457e-b62b-f390cf38ddb7"). InnerVolumeSpecName "kube-api-access-llvzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:48:21.705459 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.705396 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3cc33c45-f020-457e-b62b-f390cf38ddb7" (UID: "3cc33c45-f020-457e-b62b-f390cf38ddb7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:48:21.804450 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.804423 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llvzs\" (UniqueName: \"kubernetes.io/projected/3cc33c45-f020-457e-b62b-f390cf38ddb7-kube-api-access-llvzs\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:21.804450 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.804448 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-envoy\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:21.804657 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.804458 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:21.804657 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.804466 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-podinfo\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:21.804657 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:21.804474 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cc33c45-f020-457e-b62b-f390cf38ddb7-istio-token\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:48:22.491276 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:22.491241 2568 generic.go:356] "Generic (PLEG): container finished" podID="3cc33c45-f020-457e-b62b-f390cf38ddb7" containerID="8cc1174d68ab774b8e386b1f68c13771726bffc2d7f7ad40cd104b9a789bff7f" exitCode=0 Jul 07 13:48:22.491468 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:22.491287 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" Jul 07 13:48:22.491468 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:22.491320 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" event={"ID":"3cc33c45-f020-457e-b62b-f390cf38ddb7","Type":"ContainerDied","Data":"8cc1174d68ab774b8e386b1f68c13771726bffc2d7f7ad40cd104b9a789bff7f"} Jul 07 13:48:22.491468 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:22.491382 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf" event={"ID":"3cc33c45-f020-457e-b62b-f390cf38ddb7","Type":"ContainerDied","Data":"3672dae6698752626f9bb2daa67d5ec0ff6f4ab234baff76f0deae7c8b6ef984"} Jul 07 13:48:22.491468 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:22.491412 2568 scope.go:117] "RemoveContainer" containerID="8cc1174d68ab774b8e386b1f68c13771726bffc2d7f7ad40cd104b9a789bff7f" Jul 07 13:48:22.500855 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:22.500839 2568 scope.go:117] "RemoveContainer" containerID="b288a2252169d441a7c95a3ac7fd129ca7393b08644a0f04f6ab3e0707d851f6" Jul 07 13:48:22.508675 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:22.508659 2568 scope.go:117] "RemoveContainer" containerID="8cc1174d68ab774b8e386b1f68c13771726bffc2d7f7ad40cd104b9a789bff7f" Jul 07 13:48:22.508916 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:48:22.508898 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cc1174d68ab774b8e386b1f68c13771726bffc2d7f7ad40cd104b9a789bff7f\": container with ID starting with 8cc1174d68ab774b8e386b1f68c13771726bffc2d7f7ad40cd104b9a789bff7f not found: ID does not exist" containerID="8cc1174d68ab774b8e386b1f68c13771726bffc2d7f7ad40cd104b9a789bff7f" Jul 07 13:48:22.508957 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:22.508925 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc1174d68ab774b8e386b1f68c13771726bffc2d7f7ad40cd104b9a789bff7f"} err="failed to get container status \"8cc1174d68ab774b8e386b1f68c13771726bffc2d7f7ad40cd104b9a789bff7f\": rpc error: code = NotFound desc = could not find container \"8cc1174d68ab774b8e386b1f68c13771726bffc2d7f7ad40cd104b9a789bff7f\": container with ID starting with 8cc1174d68ab774b8e386b1f68c13771726bffc2d7f7ad40cd104b9a789bff7f not found: ID does not exist" Jul 07 13:48:22.508957 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:22.508943 2568 scope.go:117] "RemoveContainer" containerID="b288a2252169d441a7c95a3ac7fd129ca7393b08644a0f04f6ab3e0707d851f6" Jul 07 13:48:22.509168 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:48:22.509152 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b288a2252169d441a7c95a3ac7fd129ca7393b08644a0f04f6ab3e0707d851f6\": container with ID starting with b288a2252169d441a7c95a3ac7fd129ca7393b08644a0f04f6ab3e0707d851f6 not found: ID does not exist" containerID="b288a2252169d441a7c95a3ac7fd129ca7393b08644a0f04f6ab3e0707d851f6" Jul 07 13:48:22.509215 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:22.509173 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b288a2252169d441a7c95a3ac7fd129ca7393b08644a0f04f6ab3e0707d851f6"} err="failed to get container status \"b288a2252169d441a7c95a3ac7fd129ca7393b08644a0f04f6ab3e0707d851f6\": rpc error: code = NotFound desc = could not find container \"b288a2252169d441a7c95a3ac7fd129ca7393b08644a0f04f6ab3e0707d851f6\": container with ID starting with b288a2252169d441a7c95a3ac7fd129ca7393b08644a0f04f6ab3e0707d851f6 not found: ID does not exist" Jul 07 13:48:22.529978 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:22.529956 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf"] Jul 07 13:48:22.537517 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:22.537496 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-fe52df-openshift-default-7777c94957-q9wjf"] Jul 07 13:48:22.972365 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:22.972334 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cc33c45-f020-457e-b62b-f390cf38ddb7" path="/var/lib/kubelet/pods/3cc33c45-f020-457e-b62b-f390cf38ddb7/volumes" Jul 07 13:48:24.960692 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:24.960617 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:25.370276 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.370190 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:25.659721 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.659643 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8"] Jul 07 13:48:25.660380 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.660359 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cc33c45-f020-457e-b62b-f390cf38ddb7" containerName="istio-proxy" Jul 07 13:48:25.660457 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.660386 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cc33c45-f020-457e-b62b-f390cf38ddb7" containerName="istio-proxy" Jul 07 13:48:25.660544 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.660530 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cc33c45-f020-457e-b62b-f390cf38ddb7" containerName="istio-proxy" Jul 07 13:48:25.660600 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.660551 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cc33c45-f020-457e-b62b-f390cf38ddb7" containerName="istio-proxy" Jul 07 13:48:25.660600 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.660563 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2103686e-8486-408d-a135-cf5708ef0cfa" containerName="istio-proxy" Jul 07 13:48:25.664425 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.664404 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.669069 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.669050 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-2170c8-openshift-default-dockercfg-5k26h\"" Jul 07 13:48:25.688656 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.688633 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8"] Jul 07 13:48:25.740350 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.740319 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-workload-socket\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.740533 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.740369 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-envoy\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.740631 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.740545 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-workload-certs\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.740631 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.740617 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-credential-socket\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.740736 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.740654 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-data\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.740736 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.740685 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-podinfo\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.740736 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.740729 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istiod-ca-cert\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.740855 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.740809 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c8nc\" (UniqueName: \"kubernetes.io/projected/a16f619f-d408-45a0-94c6-ed829a2d9c6b-kube-api-access-4c8nc\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.740855 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.740839 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-token\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.742120 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.742097 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:25.842303 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.842263 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c8nc\" (UniqueName: \"kubernetes.io/projected/a16f619f-d408-45a0-94c6-ed829a2d9c6b-kube-api-access-4c8nc\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.842478 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.842314 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-token\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.842478 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.842355 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-workload-socket\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.842478 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.842382 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-envoy\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.842665 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.842512 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-workload-certs\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.842665 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.842562 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-credential-socket\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.842665 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.842633 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-data\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.842665 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.842660 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-podinfo\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.842858 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.842702 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istiod-ca-cert\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.842858 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.842809 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-workload-socket\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.842947 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.842870 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-data\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.842988 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.842972 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-credential-socket\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.842988 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.842982 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-workload-certs\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.843357 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.843338 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istiod-ca-cert\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.844737 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.844717 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-envoy\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.844955 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.844940 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-podinfo\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.851877 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.851860 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c8nc\" (UniqueName: \"kubernetes.io/projected/a16f619f-d408-45a0-94c6-ed829a2d9c6b-kube-api-access-4c8nc\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.853072 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.853053 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-token\") pod \"e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:25.974672 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:25.974640 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:26.151730 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:26.151705 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8"] Jul 07 13:48:26.156830 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:48:26.156778 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda16f619f_d408_45a0_94c6_ed829a2d9c6b.slice/crio-ab4c3a4ae29d340470167077ba0527e221ee6002ed9c913c433cb9a25ac395e6 WatchSource:0}: Error finding container ab4c3a4ae29d340470167077ba0527e221ee6002ed9c913c433cb9a25ac395e6: Status 404 returned error can't find the container with id ab4c3a4ae29d340470167077ba0527e221ee6002ed9c913c433cb9a25ac395e6 Jul 07 13:48:26.158763 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:26.158726 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:26.158855 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:26.158794 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:26.158855 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:26.158824 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:26.515319 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:26.515290 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" event={"ID":"a16f619f-d408-45a0-94c6-ed829a2d9c6b","Type":"ContainerStarted","Data":"b349511dd21905778f3479aca4a80037d12cf7a87c9deabe641d958882d90a51"} Jul 07 13:48:26.515501 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:26.515341 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" event={"ID":"a16f619f-d408-45a0-94c6-ed829a2d9c6b","Type":"ContainerStarted","Data":"ab4c3a4ae29d340470167077ba0527e221ee6002ed9c913c433cb9a25ac395e6"} Jul 07 13:48:26.555009 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:26.554949 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" podStartSLOduration=1.5549303559999998 podStartE2EDuration="1.554930356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:48:26.55311035 +0000 UTC m=+2064.231671680" watchObservedRunningTime="2026-07-07 13:48:26.554930356 +0000 UTC m=+2064.233491687" Jul 07 13:48:26.730905 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:26.730835 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:26.974841 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:26.974814 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:27.980146 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:27.980115 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:28.526122 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:28.526092 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:28.527099 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:28.527076 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:28.618398 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:28.618364 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:28.773246 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:28.773215 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:29.016197 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:29.016162 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:29.141566 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:29.141534 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:29.654934 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:29.654902 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:29.748568 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:29.748487 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:33.129484 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:33.129451 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:33.557685 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:33.557570 2568 generic.go:356] "Generic (PLEG): container finished" podID="9f80cafa-73f0-4027-b8c7-2a9081af5303" containerID="21d47a9b6542dbd0f4a13bb86d57195f5fca33104997c632bf84b119dc0cac01" exitCode=0 Jul 07 13:48:33.557685 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:33.557642 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" event={"ID":"9f80cafa-73f0-4027-b8c7-2a9081af5303","Type":"ContainerDied","Data":"21d47a9b6542dbd0f4a13bb86d57195f5fca33104997c632bf84b119dc0cac01"} Jul 07 13:48:33.558065 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:33.558052 2568 scope.go:117] "RemoveContainer" containerID="21d47a9b6542dbd0f4a13bb86d57195f5fca33104997c632bf84b119dc0cac01" Jul 07 13:48:33.559148 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:33.559116 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:33.559282 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:33.559205 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:33.559282 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:33.559254 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:34.473101 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:34.473065 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:34.564911 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:34.564880 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" event={"ID":"9f80cafa-73f0-4027-b8c7-2a9081af5303","Type":"ContainerStarted","Data":"2cee167a75b8f717c2cd392e54897dace1506181bd47a4177aaa41e18cecea5d"} Jul 07 13:48:34.632294 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:34.632268 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:34.633080 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:34.633053 2568 patch_prober.go:28] interesting pod/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 07 13:48:34.633209 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:34.633108 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" podUID="9f80cafa-73f0-4027-b8c7-2a9081af5303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 07 13:48:35.632294 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:35.632238 2568 patch_prober.go:28] interesting pod/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 07 13:48:35.632804 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:35.632325 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" podUID="9f80cafa-73f0-4027-b8c7-2a9081af5303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 07 13:48:36.631897 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:36.631859 2568 patch_prober.go:28] interesting pod/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 07 13:48:36.632203 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:36.632173 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" podUID="9f80cafa-73f0-4027-b8c7-2a9081af5303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 07 13:48:37.633127 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:37.633094 2568 patch_prober.go:28] interesting pod/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 07 13:48:37.633749 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:37.633707 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" podUID="9f80cafa-73f0-4027-b8c7-2a9081af5303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 07 13:48:38.632327 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:38.632294 2568 patch_prober.go:28] interesting pod/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 07 13:48:38.632619 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:38.632587 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" podUID="9f80cafa-73f0-4027-b8c7-2a9081af5303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 07 13:48:39.635930 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:39.635854 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:39.636321 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:39.636302 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:39.636919 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:39.636902 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:48:42.250901 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:42.250813 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:42.352685 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:42.352647 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:42.617678 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:42.617563 2568 generic.go:356] "Generic (PLEG): container finished" podID="a16f619f-d408-45a0-94c6-ed829a2d9c6b" containerID="b349511dd21905778f3479aca4a80037d12cf7a87c9deabe641d958882d90a51" exitCode=0 Jul 07 13:48:42.617678 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:42.617613 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" event={"ID":"a16f619f-d408-45a0-94c6-ed829a2d9c6b","Type":"ContainerDied","Data":"b349511dd21905778f3479aca4a80037d12cf7a87c9deabe641d958882d90a51"} Jul 07 13:48:42.618085 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:42.618070 2568 scope.go:117] "RemoveContainer" containerID="b349511dd21905778f3479aca4a80037d12cf7a87c9deabe641d958882d90a51" Jul 07 13:48:42.619285 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:42.619256 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:42.619369 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:42.619331 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:42.619369 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:42.619361 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:48:43.526405 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:43.526376 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:43.625496 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:43.625467 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" event={"ID":"a16f619f-d408-45a0-94c6-ed829a2d9c6b","Type":"ContainerStarted","Data":"41d8409d41deb9a2e623e47b379661e473238f11775fc9d9bf891a315991035a"} Jul 07 13:48:43.975044 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:43.975013 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:43.975914 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:43.975886 2568 patch_prober.go:28] interesting pod/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 07 13:48:43.976053 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:43.975941 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" podUID="a16f619f-d408-45a0-94c6-ed829a2d9c6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 07 13:48:44.975872 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:44.975843 2568 patch_prober.go:28] interesting pod/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 07 13:48:44.976261 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:44.975892 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" podUID="a16f619f-d408-45a0-94c6-ed829a2d9c6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 07 13:48:45.975415 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:45.975378 2568 patch_prober.go:28] interesting pod/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 07 13:48:45.975608 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:45.975434 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" podUID="a16f619f-d408-45a0-94c6-ed829a2d9c6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 07 13:48:46.975316 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:46.975288 2568 patch_prober.go:28] interesting pod/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 07 13:48:46.975712 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:46.975336 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" podUID="a16f619f-d408-45a0-94c6-ed829a2d9c6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 07 13:48:47.979233 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:47.979200 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:47.979625 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:47.979546 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:47.980227 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:47.980208 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:48:49.096365 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:49.096329 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:50.591910 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:50.591875 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:50.717397 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:50.717367 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:48:54.317989 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:48:54.317960 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:49:21.055074 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:49:21.055039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:49:24.968087 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:49:24.968055 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:50:11.134542 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:50:11.134509 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:50:11.239428 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:50:11.239401 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:50:32.773294 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:50:32.773261 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:51:04.434161 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:51:04.434130 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:51:22.550231 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:51:22.550197 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:51:23.301635 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:51:23.301601 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:51:32.342185 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:51:32.342098 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:51:33.985497 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:51:33.985465 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:51:57.575549 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:51:57.575510 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:52:03.047323 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:52:03.047291 2568 scope.go:117] "RemoveContainer" containerID="25f32b393f413e4af5b35b7bd967d48f35ba559088d969b30bd74bc84e5c0749" Jul 07 13:52:29.262000 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:52:29.261965 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:52:29.549508 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:52:29.549418 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:52:42.565670 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:52:42.565632 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:52:43.514769 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:52:43.514731 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:52:50.210629 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:52:50.210595 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:53:14.781911 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:53:14.781876 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:53:43.256022 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:53:43.255979 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:53:43.359960 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:53:43.359914 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:53:51.522014 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:53:51.521983 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:53:51.652191 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:53:51.652159 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:02.873512 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:02.873480 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:03.706519 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:03.706487 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:04.823382 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:04.823352 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:05.191609 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:05.191567 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:05.299524 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:05.299487 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:05.397440 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:05.397407 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:05.618550 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:05.618471 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:10.262669 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:10.262636 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:47.658316 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:47.658277 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:47.921352 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:47.921320 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4"] Jul 07 13:54:47.921586 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:47.921549 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" podUID="9f80cafa-73f0-4027-b8c7-2a9081af5303" containerName="istio-proxy" containerID="cri-o://2cee167a75b8f717c2cd392e54897dace1506181bd47a4177aaa41e18cecea5d" gracePeriod=30 Jul 07 13:54:47.939678 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:47.939650 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:48.192544 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:48.192454 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:48.373685 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:48.373655 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:48.783797 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:48.783761 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:48.887556 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:48.887528 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:48.982366 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:48.982334 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:49.152422 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:49.152336 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:53.178753 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.178726 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:54:53.199712 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.199655 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-credential-socket\") pod \"9f80cafa-73f0-4027-b8c7-2a9081af5303\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " Jul 07 13:54:53.199712 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.199706 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f80cafa-73f0-4027-b8c7-2a9081af5303-istiod-ca-cert\") pod \"9f80cafa-73f0-4027-b8c7-2a9081af5303\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " Jul 07 13:54:53.199864 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.199736 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-token\") pod \"9f80cafa-73f0-4027-b8c7-2a9081af5303\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " Jul 07 13:54:53.199864 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.199808 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-workload-certs\") pod \"9f80cafa-73f0-4027-b8c7-2a9081af5303\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " Jul 07 13:54:53.199864 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.199831 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-workload-socket\") pod \"9f80cafa-73f0-4027-b8c7-2a9081af5303\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " Jul 07 13:54:53.200000 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.199881 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9f80cafa-73f0-4027-b8c7-2a9081af5303" (UID: "9f80cafa-73f0-4027-b8c7-2a9081af5303"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:54:53.200000 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.199958 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kpb6\" (UniqueName: \"kubernetes.io/projected/9f80cafa-73f0-4027-b8c7-2a9081af5303-kube-api-access-4kpb6\") pod \"9f80cafa-73f0-4027-b8c7-2a9081af5303\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " Jul 07 13:54:53.200118 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.200015 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-data\") pod \"9f80cafa-73f0-4027-b8c7-2a9081af5303\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " Jul 07 13:54:53.200118 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.200060 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9f80cafa-73f0-4027-b8c7-2a9081af5303" (UID: "9f80cafa-73f0-4027-b8c7-2a9081af5303"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:54:53.200118 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.200080 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-envoy\") pod \"9f80cafa-73f0-4027-b8c7-2a9081af5303\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " Jul 07 13:54:53.200118 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.200084 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9f80cafa-73f0-4027-b8c7-2a9081af5303" (UID: "9f80cafa-73f0-4027-b8c7-2a9081af5303"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:54:53.200391 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.200112 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f80cafa-73f0-4027-b8c7-2a9081af5303-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9f80cafa-73f0-4027-b8c7-2a9081af5303" (UID: "9f80cafa-73f0-4027-b8c7-2a9081af5303"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:54:53.200391 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.200129 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-podinfo\") pod \"9f80cafa-73f0-4027-b8c7-2a9081af5303\" (UID: \"9f80cafa-73f0-4027-b8c7-2a9081af5303\") " Jul 07 13:54:53.200571 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.200554 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-credential-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:54:53.200667 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.200605 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f80cafa-73f0-4027-b8c7-2a9081af5303-istiod-ca-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:54:53.200667 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.200621 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-workload-certs\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:54:53.200667 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.200633 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-workload-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:54:53.201001 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.200978 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-data" (OuterVolumeSpecName: "istio-data") pod "9f80cafa-73f0-4027-b8c7-2a9081af5303" (UID: "9f80cafa-73f0-4027-b8c7-2a9081af5303"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:54:53.202150 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.202129 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9f80cafa-73f0-4027-b8c7-2a9081af5303" (UID: "9f80cafa-73f0-4027-b8c7-2a9081af5303"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:54:53.202150 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.202132 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f80cafa-73f0-4027-b8c7-2a9081af5303-kube-api-access-4kpb6" (OuterVolumeSpecName: "kube-api-access-4kpb6") pod "9f80cafa-73f0-4027-b8c7-2a9081af5303" (UID: "9f80cafa-73f0-4027-b8c7-2a9081af5303"). InnerVolumeSpecName "kube-api-access-4kpb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:54:53.202305 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.202287 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9f80cafa-73f0-4027-b8c7-2a9081af5303" (UID: "9f80cafa-73f0-4027-b8c7-2a9081af5303"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:54:53.202354 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.202310 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-token" (OuterVolumeSpecName: "istio-token") pod "9f80cafa-73f0-4027-b8c7-2a9081af5303" (UID: "9f80cafa-73f0-4027-b8c7-2a9081af5303"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:54:53.302098 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.302072 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-envoy\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:54:53.302098 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.302097 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-podinfo\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:54:53.302255 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.302105 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-token\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:54:53.302255 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.302114 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kpb6\" (UniqueName: \"kubernetes.io/projected/9f80cafa-73f0-4027-b8c7-2a9081af5303-kube-api-access-4kpb6\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:54:53.302255 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.302124 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f80cafa-73f0-4027-b8c7-2a9081af5303-istio-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:54:53.588662 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.588550 2568 generic.go:356] "Generic (PLEG): container finished" podID="9f80cafa-73f0-4027-b8c7-2a9081af5303" containerID="2cee167a75b8f717c2cd392e54897dace1506181bd47a4177aaa41e18cecea5d" exitCode=0 Jul 07 13:54:53.588662 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.588624 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" Jul 07 13:54:53.588662 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.588620 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" event={"ID":"9f80cafa-73f0-4027-b8c7-2a9081af5303","Type":"ContainerDied","Data":"2cee167a75b8f717c2cd392e54897dace1506181bd47a4177aaa41e18cecea5d"} Jul 07 13:54:53.588921 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.588677 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4" event={"ID":"9f80cafa-73f0-4027-b8c7-2a9081af5303","Type":"ContainerDied","Data":"ba572a2276752fbf0ba88e97ce265b02b461463725156cbeaff7445198bf1ac2"} Jul 07 13:54:53.588921 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.588699 2568 scope.go:117] "RemoveContainer" containerID="2cee167a75b8f717c2cd392e54897dace1506181bd47a4177aaa41e18cecea5d" Jul 07 13:54:53.598315 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.598300 2568 scope.go:117] "RemoveContainer" containerID="21d47a9b6542dbd0f4a13bb86d57195f5fca33104997c632bf84b119dc0cac01" Jul 07 13:54:53.606333 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.606311 2568 scope.go:117] "RemoveContainer" containerID="2cee167a75b8f717c2cd392e54897dace1506181bd47a4177aaa41e18cecea5d" Jul 07 13:54:53.606638 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:54:53.606619 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cee167a75b8f717c2cd392e54897dace1506181bd47a4177aaa41e18cecea5d\": container with ID starting with 2cee167a75b8f717c2cd392e54897dace1506181bd47a4177aaa41e18cecea5d not found: ID does not exist" containerID="2cee167a75b8f717c2cd392e54897dace1506181bd47a4177aaa41e18cecea5d" Jul 07 13:54:53.606716 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.606644 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cee167a75b8f717c2cd392e54897dace1506181bd47a4177aaa41e18cecea5d"} err="failed to get container status \"2cee167a75b8f717c2cd392e54897dace1506181bd47a4177aaa41e18cecea5d\": rpc error: code = NotFound desc = could not find container \"2cee167a75b8f717c2cd392e54897dace1506181bd47a4177aaa41e18cecea5d\": container with ID starting with 2cee167a75b8f717c2cd392e54897dace1506181bd47a4177aaa41e18cecea5d not found: ID does not exist" Jul 07 13:54:53.606716 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.606660 2568 scope.go:117] "RemoveContainer" containerID="21d47a9b6542dbd0f4a13bb86d57195f5fca33104997c632bf84b119dc0cac01" Jul 07 13:54:53.606925 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:54:53.606909 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21d47a9b6542dbd0f4a13bb86d57195f5fca33104997c632bf84b119dc0cac01\": container with ID starting with 21d47a9b6542dbd0f4a13bb86d57195f5fca33104997c632bf84b119dc0cac01 not found: ID does not exist" containerID="21d47a9b6542dbd0f4a13bb86d57195f5fca33104997c632bf84b119dc0cac01" Jul 07 13:54:53.606965 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.606930 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21d47a9b6542dbd0f4a13bb86d57195f5fca33104997c632bf84b119dc0cac01"} err="failed to get container status \"21d47a9b6542dbd0f4a13bb86d57195f5fca33104997c632bf84b119dc0cac01\": rpc error: code = NotFound desc = could not find container \"21d47a9b6542dbd0f4a13bb86d57195f5fca33104997c632bf84b119dc0cac01\": container with ID starting with 21d47a9b6542dbd0f4a13bb86d57195f5fca33104997c632bf84b119dc0cac01 not found: ID does not exist" Jul 07 13:54:53.616959 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.616937 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4"] Jul 07 13:54:53.621697 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:53.621675 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-db91fe-openshift-default-68b796d7dc-jnzc4"] Jul 07 13:54:54.970984 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:54.970951 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f80cafa-73f0-4027-b8c7-2a9081af5303" path="/var/lib/kubelet/pods/9f80cafa-73f0-4027-b8c7-2a9081af5303/volumes" Jul 07 13:54:55.047638 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:55.047610 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:55.148006 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:55.147981 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:55.300803 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:55.300704 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8"] Jul 07 13:54:55.300955 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:55.300925 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" podUID="a16f619f-d408-45a0-94c6-ed829a2d9c6b" containerName="istio-proxy" containerID="cri-o://41d8409d41deb9a2e623e47b379661e473238f11775fc9d9bf891a315991035a" gracePeriod=30 Jul 07 13:54:55.333385 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:55.333357 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:54:55.633086 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:54:55.633002 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:55:00.560118 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.560092 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:55:00.627213 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.627145 2568 generic.go:356] "Generic (PLEG): container finished" podID="a16f619f-d408-45a0-94c6-ed829a2d9c6b" containerID="41d8409d41deb9a2e623e47b379661e473238f11775fc9d9bf891a315991035a" exitCode=0 Jul 07 13:55:00.627213 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.627189 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" Jul 07 13:55:00.627213 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.627198 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" event={"ID":"a16f619f-d408-45a0-94c6-ed829a2d9c6b","Type":"ContainerDied","Data":"41d8409d41deb9a2e623e47b379661e473238f11775fc9d9bf891a315991035a"} Jul 07 13:55:00.627402 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.627249 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8" event={"ID":"a16f619f-d408-45a0-94c6-ed829a2d9c6b","Type":"ContainerDied","Data":"ab4c3a4ae29d340470167077ba0527e221ee6002ed9c913c433cb9a25ac395e6"} Jul 07 13:55:00.627402 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.627272 2568 scope.go:117] "RemoveContainer" containerID="41d8409d41deb9a2e623e47b379661e473238f11775fc9d9bf891a315991035a" Jul 07 13:55:00.636292 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.636277 2568 scope.go:117] "RemoveContainer" containerID="b349511dd21905778f3479aca4a80037d12cf7a87c9deabe641d958882d90a51" Jul 07 13:55:00.646049 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.646031 2568 scope.go:117] "RemoveContainer" containerID="41d8409d41deb9a2e623e47b379661e473238f11775fc9d9bf891a315991035a" Jul 07 13:55:00.646381 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:55:00.646352 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41d8409d41deb9a2e623e47b379661e473238f11775fc9d9bf891a315991035a\": container with ID starting with 41d8409d41deb9a2e623e47b379661e473238f11775fc9d9bf891a315991035a not found: ID does not exist" containerID="41d8409d41deb9a2e623e47b379661e473238f11775fc9d9bf891a315991035a" Jul 07 13:55:00.646463 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.646394 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41d8409d41deb9a2e623e47b379661e473238f11775fc9d9bf891a315991035a"} err="failed to get container status \"41d8409d41deb9a2e623e47b379661e473238f11775fc9d9bf891a315991035a\": rpc error: code = NotFound desc = could not find container \"41d8409d41deb9a2e623e47b379661e473238f11775fc9d9bf891a315991035a\": container with ID starting with 41d8409d41deb9a2e623e47b379661e473238f11775fc9d9bf891a315991035a not found: ID does not exist" Jul 07 13:55:00.646463 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.646417 2568 scope.go:117] "RemoveContainer" containerID="b349511dd21905778f3479aca4a80037d12cf7a87c9deabe641d958882d90a51" Jul 07 13:55:00.646735 ip-10-0-136-215 kubenswrapper[2568]: E0707 13:55:00.646716 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b349511dd21905778f3479aca4a80037d12cf7a87c9deabe641d958882d90a51\": container with ID starting with b349511dd21905778f3479aca4a80037d12cf7a87c9deabe641d958882d90a51 not found: ID does not exist" containerID="b349511dd21905778f3479aca4a80037d12cf7a87c9deabe641d958882d90a51" Jul 07 13:55:00.646825 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.646739 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b349511dd21905778f3479aca4a80037d12cf7a87c9deabe641d958882d90a51"} err="failed to get container status \"b349511dd21905778f3479aca4a80037d12cf7a87c9deabe641d958882d90a51\": rpc error: code = NotFound desc = could not find container \"b349511dd21905778f3479aca4a80037d12cf7a87c9deabe641d958882d90a51\": container with ID starting with b349511dd21905778f3479aca4a80037d12cf7a87c9deabe641d958882d90a51 not found: ID does not exist" Jul 07 13:55:00.676841 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.676820 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-envoy\") pod \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " Jul 07 13:55:00.676924 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.676873 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-credential-socket\") pod \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " Jul 07 13:55:00.676924 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.676916 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istiod-ca-cert\") pod \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " Jul 07 13:55:00.677035 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677020 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-workload-socket\") pod \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " Jul 07 13:55:00.677085 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677049 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-data\") pod \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " Jul 07 13:55:00.677140 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677089 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-podinfo\") pod \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " Jul 07 13:55:00.677140 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677112 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-workload-certs\") pod \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " Jul 07 13:55:00.677241 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677172 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4c8nc\" (UniqueName: \"kubernetes.io/projected/a16f619f-d408-45a0-94c6-ed829a2d9c6b-kube-api-access-4c8nc\") pod \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " Jul 07 13:55:00.677241 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677173 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a16f619f-d408-45a0-94c6-ed829a2d9c6b" (UID: "a16f619f-d408-45a0-94c6-ed829a2d9c6b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:55:00.677241 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677220 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-token\") pod \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\" (UID: \"a16f619f-d408-45a0-94c6-ed829a2d9c6b\") " Jul 07 13:55:00.677390 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677286 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a16f619f-d408-45a0-94c6-ed829a2d9c6b" (UID: "a16f619f-d408-45a0-94c6-ed829a2d9c6b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:55:00.677450 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677384 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a16f619f-d408-45a0-94c6-ed829a2d9c6b" (UID: "a16f619f-d408-45a0-94c6-ed829a2d9c6b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:55:00.677450 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677402 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a16f619f-d408-45a0-94c6-ed829a2d9c6b" (UID: "a16f619f-d408-45a0-94c6-ed829a2d9c6b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:55:00.677624 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677607 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-credential-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:55:00.677705 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677630 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istiod-ca-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:55:00.677705 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677644 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-workload-socket\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:55:00.677705 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677656 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-workload-certs\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:55:00.677894 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.677863 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-data" (OuterVolumeSpecName: "istio-data") pod "a16f619f-d408-45a0-94c6-ed829a2d9c6b" (UID: "a16f619f-d408-45a0-94c6-ed829a2d9c6b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:55:00.678968 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.678947 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a16f619f-d408-45a0-94c6-ed829a2d9c6b" (UID: "a16f619f-d408-45a0-94c6-ed829a2d9c6b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:55:00.679143 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.679123 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a16f619f-d408-45a0-94c6-ed829a2d9c6b" (UID: "a16f619f-d408-45a0-94c6-ed829a2d9c6b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:55:00.679202 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.679184 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a16f619f-d408-45a0-94c6-ed829a2d9c6b-kube-api-access-4c8nc" (OuterVolumeSpecName: "kube-api-access-4c8nc") pod "a16f619f-d408-45a0-94c6-ed829a2d9c6b" (UID: "a16f619f-d408-45a0-94c6-ed829a2d9c6b"). InnerVolumeSpecName "kube-api-access-4c8nc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:55:00.679319 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.679306 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-token" (OuterVolumeSpecName: "istio-token") pod "a16f619f-d408-45a0-94c6-ed829a2d9c6b" (UID: "a16f619f-d408-45a0-94c6-ed829a2d9c6b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:55:00.778191 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.778168 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:55:00.778191 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.778189 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-podinfo\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:55:00.778308 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.778198 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4c8nc\" (UniqueName: \"kubernetes.io/projected/a16f619f-d408-45a0-94c6-ed829a2d9c6b-kube-api-access-4c8nc\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:55:00.778308 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.778207 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-token\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:55:00.778308 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.778215 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a16f619f-d408-45a0-94c6-ed829a2d9c6b-istio-envoy\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 07 13:55:00.853714 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.853697 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xmrq4_7c21c845-11bc-4b8d-bb88-7f94d3edcaa3/manager/0.log" Jul 07 13:55:00.973332 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.973306 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8"] Jul 07 13:55:00.973332 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.973339 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-2170c8-openshift-default-75d8b8cfcf-g8ck8"] Jul 07 13:55:00.981996 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:00.981976 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9s7zs_cafa8c81-4732-49fa-acb1-0687968d16e6/manager/0.log" Jul 07 13:55:01.112938 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:01.112913 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-65fc57dc76-ppprg_fc812a6b-8c7e-4b2a-9e82-78aa52315c96/maas-api/0.log" Jul 07 13:55:01.503296 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:01.503271 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-777d69b966-pshl6_2f49512f-fc30-4a66-819d-7ba4a2e0a1e8/manager/0.log" Jul 07 13:55:01.995424 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:01.995400 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bdc7d5545-8p8gz_51740ce5-3d62-497a-b3ec-55ac5baf4708/copy-manifests/0.log" Jul 07 13:55:02.012581 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:02.012556 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bdc7d5545-8p8gz_51740ce5-3d62-497a-b3ec-55ac5baf4708/manager/0.log" Jul 07 13:55:02.305291 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:02.305224 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-qkhmw_149d4a33-2ae9-405b-89b8-ee816e556ebf/postgres/0.log" Jul 07 13:55:02.971160 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:02.971128 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a16f619f-d408-45a0-94c6-ed829a2d9c6b" path="/var/lib/kubelet/pods/a16f619f-d408-45a0-94c6-ed829a2d9c6b/volumes" Jul 07 13:55:03.907804 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:03.907768 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-rh22p_f438c79b-b855-4fea-94bb-e3c1affe9d6d/manager/0.log" Jul 07 13:55:04.036022 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:04.035995 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-b6t7b_311cc01e-312f-4240-8ac4-57edace3ea5f/kuadrant-console-plugin/0.log" Jul 07 13:55:04.925316 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:04.925282 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4px87z_a02dac82-c469-4f0a-a914-58459d0aec66/istio-proxy/0.log" Jul 07 13:55:05.401646 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:05.401541 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j988q_c6c35371-5324-4554-9831-222b6cde861a/istio-proxy/1.log" Jul 07 13:55:05.519551 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:05.519516 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-z95v9_b169cfbe-f84a-483e-8659-d2d720666b7b/payload-pre-processing/0.log" Jul 07 13:55:05.638467 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:05.638434 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-llz2n_0a99dba7-0297-4447-9928-91b8e05bd87b/payload-processing/0.log" Jul 07 13:55:06.104624 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:06.104595 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-xzk8s_212d7df7-0e6e-4993-a436-c84bdf87ea94/main/0.log" Jul 07 13:55:06.447503 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:06.447471 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnv5gw_8d3f206c-2eac-4918-9d73-5b83642e56be/main/0.log" Jul 07 13:55:06.559430 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:06.559404 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-s5jhf_d0aa835b-3bc1-45ad-971b-5d89199d7d91/main/0.log" Jul 07 13:55:10.975311 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:10.975277 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:55:11.089813 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:11.089774 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:55:13.989543 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:13.989513 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j2rqx_730e3c49-83b7-4a1d-9b7c-d5621955f6df/global-pull-secret-syncer/0.log" Jul 07 13:55:14.124856 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:14.124827 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" Jul 07 13:55:14.314413 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:14.314329 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-215.ec2.internal_c861ca34efcd36b7887620391a65da0e/haproxy/0.log" Jul 07 13:55:19.397988 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:19.397962 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-rh22p_f438c79b-b855-4fea-94bb-e3c1affe9d6d/manager/0.log" Jul 07 13:55:19.440118 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:19.440093 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-b6t7b_311cc01e-312f-4240-8ac4-57edace3ea5f/kuadrant-console-plugin/0.log" Jul 07 13:55:21.336671 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.336594 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3e8df037-10ed-46f6-abd6-472d9ed3c1ef/alertmanager/0.log" Jul 07 13:55:21.366768 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.366743 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3e8df037-10ed-46f6-abd6-472d9ed3c1ef/config-reloader/0.log" Jul 07 13:55:21.403169 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.403149 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3e8df037-10ed-46f6-abd6-472d9ed3c1ef/kube-rbac-proxy-web/0.log" Jul 07 13:55:21.432975 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.432952 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3e8df037-10ed-46f6-abd6-472d9ed3c1ef/kube-rbac-proxy/0.log" Jul 07 13:55:21.458085 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.458065 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3e8df037-10ed-46f6-abd6-472d9ed3c1ef/kube-rbac-proxy-metric/0.log" Jul 07 13:55:21.485840 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.485812 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3e8df037-10ed-46f6-abd6-472d9ed3c1ef/prom-label-proxy/0.log" Jul 07 13:55:21.517390 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.517363 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3e8df037-10ed-46f6-abd6-472d9ed3c1ef/init-config-reloader/0.log" Jul 07 13:55:21.617399 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.617326 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-tmbbp_0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0/kube-state-metrics/0.log" Jul 07 13:55:21.650199 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.650175 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-tmbbp_0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0/kube-rbac-proxy-main/0.log" Jul 07 13:55:21.677450 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.677429 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-tmbbp_0ea7c55e-4cdf-4cc5-8e6c-0da0af9ad5a0/kube-rbac-proxy-self/0.log" Jul 07 13:55:21.710353 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.710328 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-c8557b76-7gc7w_79e96c1a-298b-465b-9709-c203521ae225/metrics-server/0.log" Jul 07 13:55:21.746130 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.746106 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-fdn9n_54d7b121-70f9-41b2-8a1c-5e5ff2f37533/monitoring-plugin/0.log" Jul 07 13:55:21.779980 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.779958 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7vrl2_d16107fb-6648-4083-ab38-dc0f21497f5a/node-exporter/0.log" Jul 07 13:55:21.805405 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.805381 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7vrl2_d16107fb-6648-4083-ab38-dc0f21497f5a/kube-rbac-proxy/0.log" Jul 07 13:55:21.832702 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:21.832654 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7vrl2_d16107fb-6648-4083-ab38-dc0f21497f5a/init-textfile/0.log" Jul 07 13:55:22.060735 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.060705 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-xb8hz_ffac38b9-3729-4e85-8128-ebf688f659fd/kube-rbac-proxy-main/0.log" Jul 07 13:55:22.087461 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.087440 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-xb8hz_ffac38b9-3729-4e85-8128-ebf688f659fd/kube-rbac-proxy-self/0.log" Jul 07 13:55:22.109414 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.109392 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-xb8hz_ffac38b9-3729-4e85-8128-ebf688f659fd/openshift-state-metrics/0.log" Jul 07 13:55:22.166358 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.166336 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_505e1ce2-f48e-463b-9eae-b6ae88f14234/prometheus/0.log" Jul 07 13:55:22.189724 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.189701 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_505e1ce2-f48e-463b-9eae-b6ae88f14234/config-reloader/0.log" Jul 07 13:55:22.218159 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.218136 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_505e1ce2-f48e-463b-9eae-b6ae88f14234/thanos-sidecar/0.log" Jul 07 13:55:22.246098 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.246076 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_505e1ce2-f48e-463b-9eae-b6ae88f14234/kube-rbac-proxy-web/0.log" Jul 07 13:55:22.271414 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.271393 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_505e1ce2-f48e-463b-9eae-b6ae88f14234/kube-rbac-proxy/0.log" Jul 07 13:55:22.298229 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.298197 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_505e1ce2-f48e-463b-9eae-b6ae88f14234/kube-rbac-proxy-thanos/0.log" Jul 07 13:55:22.320021 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.319984 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_505e1ce2-f48e-463b-9eae-b6ae88f14234/init-config-reloader/0.log" Jul 07 13:55:22.351163 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.351103 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-vqwpp_fc0d4d82-c529-48b8-b7ef-156bec15b216/prometheus-operator/0.log" Jul 07 13:55:22.375441 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.375419 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-vqwpp_fc0d4d82-c529-48b8-b7ef-156bec15b216/kube-rbac-proxy/0.log" Jul 07 13:55:22.444055 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.443987 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-747c7c65d5-kbz84_b5062415-d6c0-49c3-a582-28b98b6e5faf/telemeter-client/0.log" Jul 07 13:55:22.466126 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.466102 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-747c7c65d5-kbz84_b5062415-d6c0-49c3-a582-28b98b6e5faf/reload/0.log" Jul 07 13:55:22.492772 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.492749 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-747c7c65d5-kbz84_b5062415-d6c0-49c3-a582-28b98b6e5faf/kube-rbac-proxy/0.log" Jul 07 13:55:22.520524 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.520502 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7566859584-k6vrm_33c98f55-320a-437c-b136-60180fc3ff4e/thanos-query/0.log" Jul 07 13:55:22.542161 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.542139 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7566859584-k6vrm_33c98f55-320a-437c-b136-60180fc3ff4e/kube-rbac-proxy-web/0.log" Jul 07 13:55:22.565728 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.565710 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7566859584-k6vrm_33c98f55-320a-437c-b136-60180fc3ff4e/kube-rbac-proxy/0.log" Jul 07 13:55:22.587456 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.587437 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7566859584-k6vrm_33c98f55-320a-437c-b136-60180fc3ff4e/prom-label-proxy/0.log" Jul 07 13:55:22.612019 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.611996 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7566859584-k6vrm_33c98f55-320a-437c-b136-60180fc3ff4e/kube-rbac-proxy-rules/0.log" Jul 07 13:55:22.645281 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:22.645254 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7566859584-k6vrm_33c98f55-320a-437c-b136-60180fc3ff4e/kube-rbac-proxy-metrics/0.log" Jul 07 13:55:23.619025 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.618991 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn"] Jul 07 13:55:23.619797 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.619766 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f80cafa-73f0-4027-b8c7-2a9081af5303" containerName="istio-proxy" Jul 07 13:55:23.619797 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.619796 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f80cafa-73f0-4027-b8c7-2a9081af5303" containerName="istio-proxy" Jul 07 13:55:23.619943 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.619806 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f80cafa-73f0-4027-b8c7-2a9081af5303" containerName="istio-proxy" Jul 07 13:55:23.619943 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.619813 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f80cafa-73f0-4027-b8c7-2a9081af5303" containerName="istio-proxy" Jul 07 13:55:23.619943 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.619830 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cc33c45-f020-457e-b62b-f390cf38ddb7" containerName="istio-proxy" Jul 07 13:55:23.619943 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.619835 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cc33c45-f020-457e-b62b-f390cf38ddb7" containerName="istio-proxy" Jul 07 13:55:23.619943 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.619854 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a16f619f-d408-45a0-94c6-ed829a2d9c6b" containerName="istio-proxy" Jul 07 13:55:23.619943 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.619862 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a16f619f-d408-45a0-94c6-ed829a2d9c6b" containerName="istio-proxy" Jul 07 13:55:23.619943 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.619879 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a16f619f-d408-45a0-94c6-ed829a2d9c6b" containerName="istio-proxy" Jul 07 13:55:23.619943 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.619888 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a16f619f-d408-45a0-94c6-ed829a2d9c6b" containerName="istio-proxy" Jul 07 13:55:23.620200 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.619989 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a16f619f-d408-45a0-94c6-ed829a2d9c6b" containerName="istio-proxy" Jul 07 13:55:23.620200 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.620005 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f80cafa-73f0-4027-b8c7-2a9081af5303" containerName="istio-proxy" Jul 07 13:55:23.620200 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.620015 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f80cafa-73f0-4027-b8c7-2a9081af5303" containerName="istio-proxy" Jul 07 13:55:23.622337 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.622319 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.625459 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.625438 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g27ln\"/\"openshift-service-ca.crt\"" Jul 07 13:55:23.625747 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.625726 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g27ln\"/\"kube-root-ca.crt\"" Jul 07 13:55:23.626843 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.626825 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g27ln\"/\"default-dockercfg-vwth7\"" Jul 07 13:55:23.638965 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.638937 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn"] Jul 07 13:55:23.707984 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.707950 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-podres\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.708144 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.707999 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-lib-modules\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.708144 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.708061 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-proc\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.708144 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.708108 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rxr2\" (UniqueName: \"kubernetes.io/projected/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-kube-api-access-9rxr2\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.708144 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.708140 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-sys\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.808972 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.808941 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rxr2\" (UniqueName: \"kubernetes.io/projected/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-kube-api-access-9rxr2\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.809130 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.808988 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-sys\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.809130 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.809042 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-podres\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.809130 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.809072 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-lib-modules\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.809130 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.809107 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-proc\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.809312 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.809132 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-sys\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.809312 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.809205 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-podres\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.809312 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.809217 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-proc\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.809312 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.809220 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-lib-modules\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.818112 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.818084 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rxr2\" (UniqueName: \"kubernetes.io/projected/99ee9a0a-9a13-4ccc-a906-cb9daaf434d1-kube-api-access-9rxr2\") pod \"perf-node-gather-daemonset-txbmn\" (UID: \"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:23.935767 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:23.935732 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:24.080322 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:24.080298 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn"] Jul 07 13:55:24.082699 ip-10-0-136-215 kubenswrapper[2568]: W0707 13:55:24.082661 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod99ee9a0a_9a13_4ccc_a906_cb9daaf434d1.slice/crio-6654e278f812ec16439cf737f58b3fe0d7bac260478e5d6b4fb202a067da00d5 WatchSource:0}: Error finding container 6654e278f812ec16439cf737f58b3fe0d7bac260478e5d6b4fb202a067da00d5: Status 404 returned error can't find the container with id 6654e278f812ec16439cf737f58b3fe0d7bac260478e5d6b4fb202a067da00d5 Jul 07 13:55:24.751595 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:24.751546 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" event={"ID":"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1","Type":"ContainerStarted","Data":"760782d370bed0eb2691d8de5305dfcdd07ca95a514003dc7ac4f1a95384cf75"} Jul 07 13:55:24.751954 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:24.751611 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" event={"ID":"99ee9a0a-9a13-4ccc-a906-cb9daaf434d1","Type":"ContainerStarted","Data":"6654e278f812ec16439cf737f58b3fe0d7bac260478e5d6b4fb202a067da00d5"} Jul 07 13:55:24.751954 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:24.751694 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:24.777400 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:24.777349 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" podStartSLOduration=1.777335101 podStartE2EDuration="1.777335101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:55:24.776405129 +0000 UTC m=+2482.454966458" watchObservedRunningTime="2026-07-07 13:55:24.777335101 +0000 UTC m=+2482.455896432" Jul 07 13:55:24.958049 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:24.958023 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5797dc48b7-8jsrb_4345a50e-990d-4242-a0f1-948b1257f288/console/0.log" Jul 07 13:55:26.629835 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:26.629804 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qzrm2_7e1f2bac-745c-46b5-8a4b-b745449eb8e2/dns/0.log" Jul 07 13:55:26.653030 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:26.653000 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qzrm2_7e1f2bac-745c-46b5-8a4b-b745449eb8e2/kube-rbac-proxy/0.log" Jul 07 13:55:26.854208 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:26.854176 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q9ck9_7416c196-b925-447d-8a35-a5c5421d15d1/dns-node-resolver/0.log" Jul 07 13:55:27.524803 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:27.524773 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z856v_0bbd3146-8a83-4cdd-9b41-410b53b540f4/node-ca/0.log" Jul 07 13:55:28.586048 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:28.586020 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4px87z_a02dac82-c469-4f0a-a914-58459d0aec66/istio-proxy/0.log" Jul 07 13:55:28.928544 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:28.928513 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j988q_c6c35371-5324-4554-9831-222b6cde861a/istio-proxy/0.log" Jul 07 13:55:28.953183 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:28.953155 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j988q_c6c35371-5324-4554-9831-222b6cde861a/istio-proxy/1.log" Jul 07 13:55:28.985385 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:28.985362 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-z95v9_b169cfbe-f84a-483e-8659-d2d720666b7b/payload-pre-processing/0.log" Jul 07 13:55:29.043276 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:29.043232 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-llz2n_0a99dba7-0297-4447-9928-91b8e05bd87b/payload-processing/0.log" Jul 07 13:55:29.769187 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:29.769163 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9zqg5_627d53fa-7581-4f07-94ee-78932d124b75/serve-healthcheck-canary/0.log" Jul 07 13:55:30.362656 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:30.362627 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-25qdm_faf07002-635a-4f2b-a452-be3e757a9d2f/kube-rbac-proxy/0.log" Jul 07 13:55:30.390173 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:30.390146 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-25qdm_faf07002-635a-4f2b-a452-be3e757a9d2f/exporter/0.log" Jul 07 13:55:30.417285 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:30.417259 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-25qdm_faf07002-635a-4f2b-a452-be3e757a9d2f/extractor/0.log" Jul 07 13:55:30.767478 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:30.767447 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-txbmn" Jul 07 13:55:33.165468 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:33.165443 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xmrq4_7c21c845-11bc-4b8d-bb88-7f94d3edcaa3/manager/0.log" Jul 07 13:55:33.291233 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:33.291207 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9s7zs_cafa8c81-4732-49fa-acb1-0687968d16e6/manager/0.log" Jul 07 13:55:33.338923 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:33.338900 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-65fc57dc76-ppprg_fc812a6b-8c7e-4b2a-9e82-78aa52315c96/maas-api/0.log" Jul 07 13:55:33.507870 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:33.507791 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-777d69b966-pshl6_2f49512f-fc30-4a66-819d-7ba4a2e0a1e8/manager/0.log" Jul 07 13:55:33.906215 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:33.906127 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4px87z_a02dac82-c469-4f0a-a914-58459d0aec66/istio-proxy/0.log" Jul 07 13:55:33.922584 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:33.922556 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bdc7d5545-8p8gz_51740ce5-3d62-497a-b3ec-55ac5baf4708/manager/0.log" Jul 07 13:55:33.973548 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:33.973529 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bdc7d5545-8p8gz_51740ce5-3d62-497a-b3ec-55ac5baf4708/copy-manifests/0.log" Jul 07 13:55:34.195532 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:34.195509 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j988q_c6c35371-5324-4554-9831-222b6cde861a/istio-proxy/0.log" Jul 07 13:55:34.209398 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:34.209369 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-qkhmw_149d4a33-2ae9-405b-89b8-ee816e556ebf/postgres/0.log" Jul 07 13:55:34.222637 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:34.222610 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j988q_c6c35371-5324-4554-9831-222b6cde861a/istio-proxy/1.log" Jul 07 13:55:34.245136 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:34.245111 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-z95v9_b169cfbe-f84a-483e-8659-d2d720666b7b/payload-pre-processing/0.log" Jul 07 13:55:34.287113 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:34.287089 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-llz2n_0a99dba7-0297-4447-9928-91b8e05bd87b/payload-processing/0.log" Jul 07 13:55:35.629053 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:35.629025 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b44df4f55-ndw7v_93fa4a10-6ee4-44f8-be51-5064bd32061d/manager/0.log" Jul 07 13:55:35.629534 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:35.629518 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4px87z_a02dac82-c469-4f0a-a914-58459d0aec66/istio-proxy/0.log" Jul 07 13:55:36.039296 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:36.039262 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j988q_c6c35371-5324-4554-9831-222b6cde861a/istio-proxy/0.log" Jul 07 13:55:36.065763 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:36.065733 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-j988q_c6c35371-5324-4554-9831-222b6cde861a/istio-proxy/1.log" Jul 07 13:55:36.091260 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:36.091231 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-z95v9_b169cfbe-f84a-483e-8659-d2d720666b7b/payload-pre-processing/0.log" Jul 07 13:55:36.131607 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:36.131563 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-llz2n_0a99dba7-0297-4447-9928-91b8e05bd87b/payload-processing/0.log" Jul 07 13:55:37.644148 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:37.644113 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024 Jul 07 13:55:37.743409 ip-10-0-136-215 kubenswrapper[2568]: I0707 13:55:37.743382 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hm7dm_331b85aa-4b8d-4efb-bcdb-4cec4bdae545/konnectivity-agent/0.log" limit=1024