Jul 08 19:11:31.217671 ip-10-0-128-253 systemd[1]: Starting Kubernetes Kubelet... Jul 08 19:11:31.652246 ip-10-0-128-253 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:11:31.652246 ip-10-0-128-253 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 19:11:31.652246 ip-10-0-128-253 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:11:31.652246 ip-10-0-128-253 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 19:11:31.652246 ip-10-0-128-253 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:11:31.652246 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.621675 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 19:11:31.652246 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627231 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:11:31.652246 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627244 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:11:31.652246 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627249 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:11:31.652246 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627252 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:11:31.652246 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627255 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:11:31.652246 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627259 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:11:31.652246 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627263 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:11:31.652246 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627266 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627270 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627273 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627276 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627279 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627282 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627285 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627289 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627291 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627294 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627297 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627300 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627303 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627306 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627308 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627311 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627314 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627317 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627328 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:11:31.653201 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627333 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627337 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627341 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627344 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627348 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627351 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627353 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627356 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627359 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627362 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627364 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627367 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627370 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627373 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627376 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627378 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627381 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627384 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627387 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:11:31.653914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627389 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627392 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627394 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627397 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627400 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627402 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627405 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627408 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627410 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627413 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627416 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627418 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627421 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627424 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627427 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627430 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627434 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627438 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627442 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627445 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:11:31.654494 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627460 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627464 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627467 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627470 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627472 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627476 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627479 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627481 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627485 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627488 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627491 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627493 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627496 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627499 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627502 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627505 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627507 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627510 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627513 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:11:31.655162 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627515 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627518 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627521 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627524 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627527 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627529 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627534 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627537 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627540 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627543 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627546 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627549 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627552 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627555 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.627557 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628646 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628662 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628670 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628675 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628680 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:11:31.655836 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628697 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628702 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628707 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628711 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628716 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628727 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628732 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628737 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628742 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628747 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628751 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628755 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628760 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628764 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628768 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628773 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628777 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628782 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628792 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628796 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:11:31.656773 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628802 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628807 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628811 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628816 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628820 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628824 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628829 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628834 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628838 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628842 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628847 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628857 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628861 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628866 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628872 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628877 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628881 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628888 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628894 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628899 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:11:31.657746 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628904 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628908 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628918 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628922 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628928 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628933 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628937 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628941 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628945 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628952 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628956 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628961 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628966 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628975 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628980 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628984 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628989 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628993 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.628997 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629002 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:11:31.658359 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629007 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629011 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629016 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629020 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629024 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629029 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629039 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629044 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629048 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629053 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629058 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629063 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629067 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629071 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629075 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629080 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629084 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629089 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629093 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629103 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:11:31.659195 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629110 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629115 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629120 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629124 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629129 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629134 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629139 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629143 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629147 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629151 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629157 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629167 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629171 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.629183 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630272 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630285 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630289 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630293 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630296 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630299 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:11:31.659758 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630302 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630305 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630308 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630311 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630314 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630317 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630320 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630324 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630328 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630332 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630335 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630338 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630341 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630344 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630346 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630349 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630352 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630355 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630358 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:11:31.660272 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630361 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630364 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630366 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630369 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630373 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630377 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630380 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630384 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630387 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630390 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630393 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630395 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630398 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630400 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630403 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630406 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630409 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630411 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630414 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630417 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:11:31.660806 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630419 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630422 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630425 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630427 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630430 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630454 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630457 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630460 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630462 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630465 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630468 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630471 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630474 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630476 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630479 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630484 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630488 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630491 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630499 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:11:31.661397 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630502 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630505 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630509 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630512 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630515 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630518 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630521 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630524 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630527 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630530 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630532 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630535 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630538 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630540 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630543 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630546 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630548 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630551 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630554 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:11:31.661929 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630557 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630559 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630562 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630565 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630567 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630570 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630574 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630576 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630579 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630582 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630584 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630587 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630592 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630594 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630597 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.630600 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630666 2563 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630673 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630680 2563 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630687 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630693 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 19:11:31.662424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630698 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630705 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630716 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630720 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630723 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630727 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630730 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630733 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630736 2563 flags.go:64] FLAG: --cgroup-root="" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630739 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630742 2563 flags.go:64] FLAG: --client-ca-file="" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630745 2563 flags.go:64] FLAG: --cloud-provider="external" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630748 2563 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630753 2563 flags.go:64] FLAG: --cluster-domain="" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630756 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630759 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630763 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630766 2563 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630770 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630773 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630776 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630780 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630783 2563 flags.go:64] FLAG: --contention-profiling="false" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630788 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 19:11:31.663036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630791 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630794 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630798 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630802 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630805 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630808 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630811 2563 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630814 2563 flags.go:64] FLAG: --enable-server="true" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630817 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630823 2563 flags.go:64] FLAG: --event-burst="100" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630826 2563 flags.go:64] FLAG: --event-qps="50" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630829 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630833 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630836 2563 flags.go:64] FLAG: --eviction-hard="" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630840 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630843 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630846 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630849 2563 flags.go:64] FLAG: --eviction-soft="" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630852 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630855 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630858 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630861 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630864 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630867 2563 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630870 2563 flags.go:64] FLAG: --feature-gates="" Jul 08 19:11:31.663720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630874 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630877 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630880 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630884 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630887 2563 flags.go:64] FLAG: --healthz-port="10248" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630890 2563 flags.go:64] FLAG: --help="false" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630894 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-128-253.ec2.internal" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630898 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630902 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630905 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630909 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630912 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630915 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630918 2563 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630922 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630925 2563 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630928 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630932 2563 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630935 2563 flags.go:64] FLAG: --kube-reserved="" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630938 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630941 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630944 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630947 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630950 2563 flags.go:64] FLAG: --lock-file="" Jul 08 19:11:31.664624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630953 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630956 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630959 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630965 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630968 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630971 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630974 2563 flags.go:64] FLAG: --logging-format="text" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630977 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630981 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630984 2563 flags.go:64] FLAG: --manifest-url="" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630987 2563 flags.go:64] FLAG: --manifest-url-header="" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630991 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630995 2563 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.630999 2563 flags.go:64] FLAG: --max-pods="110" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631004 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631007 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631011 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631015 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631018 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631021 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631024 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631031 2563 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631034 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631037 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 19:11:31.665468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631041 2563 flags.go:64] FLAG: --pod-cidr="" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631044 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631050 2563 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631053 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631056 2563 flags.go:64] FLAG: --pods-per-core="0" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631059 2563 flags.go:64] FLAG: --port="10250" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631063 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631066 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09109912282f71c6c" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631069 2563 flags.go:64] FLAG: --qos-reserved="" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631072 2563 flags.go:64] FLAG: --read-only-port="10255" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631075 2563 flags.go:64] FLAG: --register-node="true" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631078 2563 flags.go:64] FLAG: --register-with-taints="" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631082 2563 flags.go:64] FLAG: --registry-burst="10" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631087 2563 flags.go:64] FLAG: --registry-qps="5" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631090 2563 flags.go:64] FLAG: --reserved-cpus="" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631093 2563 flags.go:64] FLAG: --reserved-memory="" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631097 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631100 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631103 2563 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631106 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631109 2563 flags.go:64] FLAG: --runonce="false" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631112 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631115 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631119 2563 flags.go:64] FLAG: --seccomp-default="false" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631124 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 19:11:31.666097 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631127 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631130 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631134 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631137 2563 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631140 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631143 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631146 2563 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631149 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631152 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631155 2563 flags.go:64] FLAG: --system-cgroups="" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631158 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631163 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631166 2563 flags.go:64] FLAG: --tls-cert-file="" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631169 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631173 2563 flags.go:64] FLAG: --tls-min-version="" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631176 2563 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631179 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631182 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631185 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631188 2563 flags.go:64] FLAG: --v="2" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631192 2563 flags.go:64] FLAG: --version="false" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631197 2563 flags.go:64] FLAG: --vmodule="" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631201 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.631204 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631322 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:11:31.666936 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631326 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631329 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631332 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631335 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631338 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631342 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631347 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631350 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631352 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631355 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631358 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631361 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631363 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631366 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631369 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631372 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631374 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631377 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631380 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631383 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:11:31.667736 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631386 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631388 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631391 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631393 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631396 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631399 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631401 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631404 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631407 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631410 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631413 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631416 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631419 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631421 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631424 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631427 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631431 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631436 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631440 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631443 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:11:31.668270 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631445 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631463 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631466 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631469 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631472 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631475 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631478 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631481 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631483 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631486 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631489 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631492 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631495 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631498 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631500 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631503 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631506 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631509 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631511 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631514 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:11:31.668842 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631517 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631520 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631526 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631530 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631533 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631536 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631539 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631542 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631545 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631549 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631553 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631556 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631559 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631562 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631564 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631569 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631572 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631575 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631578 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:11:31.669364 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631581 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631583 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631586 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631589 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631592 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631594 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631597 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631600 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631603 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631605 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631608 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631610 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631613 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631616 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631618 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631621 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631625 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631628 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:11:31.669870 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.631631 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:11:31.670338 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.632117 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:11:31.670338 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.639528 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 19:11:31.670338 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.639545 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 19:11:31.670338 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.640503 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 19:11:31.670338 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.640515 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 19:11:31.670338 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640573 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:11:31.670338 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640578 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:11:31.670338 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640582 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:11:31.670338 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640585 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:11:31.670338 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640588 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:11:31.670338 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640593 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:11:31.670338 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640597 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:11:31.670338 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640600 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:11:31.670338 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640604 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640607 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640610 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640613 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640617 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640620 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640623 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640626 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640629 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640631 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640634 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640637 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640639 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640642 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640645 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640648 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640650 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640653 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640656 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640659 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:11:31.670747 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640662 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640666 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640668 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640671 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640674 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640677 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640680 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640683 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640686 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640689 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640691 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640694 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640697 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640700 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640703 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640707 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640710 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640714 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640717 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640719 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:11:31.671319 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640722 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640725 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640728 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640731 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640735 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640738 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640740 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640744 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640747 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640750 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640753 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640756 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640759 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640761 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640765 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640768 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640771 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640773 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640777 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640779 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:11:31.671979 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640782 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640785 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640787 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640790 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640792 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640795 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640798 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640800 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640803 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640806 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640808 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640811 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640814 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640818 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640822 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640825 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640828 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640830 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640833 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640836 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:11:31.672878 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640839 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640842 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640845 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640848 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640851 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640854 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640856 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640860 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640863 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640866 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640868 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.640873 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640986 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640992 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640995 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:11:31.673768 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.640998 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641001 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641004 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641007 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641010 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641013 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641016 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641018 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641021 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641023 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641027 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641030 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641033 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641035 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641038 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641040 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641046 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641049 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641054 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:11:31.674580 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641057 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641059 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641063 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641066 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641068 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641071 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641074 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641077 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641080 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641084 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641088 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641091 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641095 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641097 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641100 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641103 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641106 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641109 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641112 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:11:31.675509 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641116 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641119 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641122 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641125 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641128 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641131 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641134 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641136 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641139 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641142 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641146 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641149 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641155 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641158 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641161 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641164 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641167 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641169 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641172 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641174 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:11:31.676461 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641181 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641184 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641186 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641189 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641192 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641196 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641200 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641203 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641206 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641209 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641212 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641215 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641218 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641221 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641224 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641226 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641229 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641232 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641234 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:11:31.677419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641237 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641240 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641242 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641247 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641250 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641254 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641257 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641260 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641263 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641266 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641268 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641271 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641274 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641277 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641280 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641282 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641285 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641288 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:11:31.678303 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:31.641291 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:11:31.679216 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.641296 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:11:31.679216 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.641431 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 19:11:31.679216 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.643598 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 19:11:31.679216 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.645177 2563 server.go:1019] "Starting client certificate rotation" Jul 08 19:11:31.679216 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.645298 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 19:11:31.679216 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.645336 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 19:11:31.679216 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.648406 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 19:11:31.679216 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.651490 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 19:11:31.679216 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.660452 2563 log.go:25] "Validated CRI v1 runtime API" Jul 08 19:11:31.679216 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.662521 2563 log.go:25] "Validated CRI v1 image API" Jul 08 19:11:31.679216 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.667050 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 19:11:31.679216 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.672215 2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 e049bbe3-accc-4d73-8aaa-9479ff88871e:/dev/nvme0n1p4 f00a4f5d-9e10-4217-8934-fdfb52dae0c3:/dev/nvme0n1p3] Jul 08 19:11:31.679564 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.672236 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 19:11:31.679564 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.678895 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 19:11:31.680702 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.680584 2563 manager.go:216] Machine: {Timestamp:2026-07-08 19:11:31.67775661 +0000 UTC m=+0.347008295 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096196 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2446577a2bdf04c7a356ced0f526bd SystemUUID:ec244657-7a2b-df04-c7a3-56ced0f526bd BootID:f3400d09-0962-437c-8c55-01adf40a0d83 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b7:f0:43:47:75 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b7:f0:43:47:75 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:9e:76:ae:46:fe Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 19:11:31.680702 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.680701 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 19:11:31.680849 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.680815 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 19:11:31.683830 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.683799 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 19:11:31.684006 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.683833 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-253.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 19:11:31.684653 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.684640 2563 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 19:11:31.684720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.684659 2563 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 19:11:31.684720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.684670 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 19:11:31.685300 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.685284 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 19:11:31.686577 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.686564 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 19:11:31.689755 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.689740 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 08 19:11:31.689886 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.689874 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 19:11:31.689945 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.689913 2563 kubelet.go:486] "Attempting to sync node with API server" Jul 08 19:11:31.689945 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.689936 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 19:11:31.690048 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.689957 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 19:11:31.690048 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.689972 2563 kubelet.go:398] "Adding apiserver pod source" Jul 08 19:11:31.690048 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.689986 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 19:11:31.690379 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.690354 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 19:11:31.690444 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.690390 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 19:11:31.694052 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.694032 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 19:11:31.695986 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.695969 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 19:11:31.696076 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.696009 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 19:11:31.697234 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.697220 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 19:11:31.697314 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.697241 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 19:11:31.697314 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.697251 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 19:11:31.697314 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.697261 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 19:11:31.697314 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.697274 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 19:11:31.697314 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.697283 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 19:11:31.697314 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.697302 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 19:11:31.697314 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.697312 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 19:11:31.697578 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.697323 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 19:11:31.697578 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.697333 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 19:11:31.697578 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.697346 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 19:11:31.697578 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.697355 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 19:11:31.698198 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.698187 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 19:11:31.698254 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.698201 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 19:11:31.701398 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.701381 2563 server.go:1268] "Started kubelet" Jul 08 19:11:31.701523 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.701491 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 19:11:31.701625 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.701577 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 19:11:31.701680 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.701652 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 19:11:31.705431 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.705403 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 19:11:31.705741 ip-10-0-128-253 systemd[1]: Started Kubernetes Kubelet. Jul 08 19:11:31.707760 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:31.705799 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 19:11:31.707760 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.705872 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-253.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 19:11:31.707760 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:31.705938 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-253.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 19:11:31.707760 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.706059 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z7rm8" Jul 08 19:11:31.708583 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:31.708558 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 19:11:31.708884 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.708869 2563 server.go:310] "Adding debug handlers to kubelet server" Jul 08 19:11:31.709150 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.709132 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z7rm8" Jul 08 19:11:31.712581 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.712558 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 19:11:31.712668 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.712559 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 19:11:31.713104 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:31.713088 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:31.713325 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.713305 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 19:11:31.713474 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.713458 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 19:11:31.713566 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:31.711857 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-253.ec2.internal.18c066434e4ee1ab default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-253.ec2.internal,UID:ip-10-0-128-253.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-253.ec2.internal,},FirstTimestamp:2026-07-08 19:11:31.701350827 +0000 UTC m=+0.370602510,LastTimestamp:2026-07-08 19:11:31.701350827 +0000 UTC m=+0.370602510,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-253.ec2.internal,}" Jul 08 19:11:31.713566 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.713483 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 19:11:31.713733 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.713560 2563 factory.go:55] Registering systemd factory Jul 08 19:11:31.713733 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.713629 2563 factory.go:223] Registration of the systemd container factory successfully Jul 08 19:11:31.713733 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.713685 2563 reconstruct.go:97] "Volume reconstruction finished" Jul 08 19:11:31.713733 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.713698 2563 reconciler.go:29] "Reconciler: start to sync state" Jul 08 19:11:31.713953 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.713937 2563 factory.go:153] Registering CRI-O factory Jul 08 19:11:31.714011 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.713955 2563 factory.go:223] Registration of the crio container factory successfully Jul 08 19:11:31.714011 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.714007 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 19:11:31.714091 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.714031 2563 factory.go:103] Registering Raw factory Jul 08 19:11:31.714091 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.714046 2563 manager.go:1196] Started watching for new ooms in manager Jul 08 19:11:31.714458 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.714430 2563 manager.go:349] Starting recovery of all containers Jul 08 19:11:31.722676 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.722656 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:11:31.723981 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.723965 2563 manager.go:354] Recovery completed Jul 08 19:11:31.725913 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:31.725890 2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-253.ec2.internal\" not found" node="ip-10-0-128-253.ec2.internal" Jul 08 19:11:31.729895 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.729881 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:11:31.731776 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.731760 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:11:31.731854 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.731791 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:11:31.731854 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.731806 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:11:31.732316 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.732299 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 19:11:31.732316 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.732313 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 19:11:31.732488 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.732326 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:11:31.732488 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.732331 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 08 19:11:31.733118 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.733096 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:11:31.733183 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.733134 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:11:31.733183 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.733148 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:11:31.734878 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.734865 2563 policy_none.go:49] "None policy: Start" Jul 08 19:11:31.734923 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.734883 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 19:11:31.734923 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.734895 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 19:11:31.736116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.736007 2563 policy_none.go:47] "Start" Jul 08 19:11:31.777044 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.777024 2563 manager.go:337] "Starting Device Plugin manager" Jul 08 19:11:31.777116 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:31.777059 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 19:11:31.777116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.777075 2563 server.go:87] "Starting device plugin registration server" Jul 08 19:11:31.777361 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.777345 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 19:11:31.777441 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.777363 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 19:11:31.777556 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.777537 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 19:11:31.777642 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.777625 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 19:11:31.777642 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.777643 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 19:11:31.778223 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:31.778202 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 19:11:31.778287 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:31.778266 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:31.881678 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.881637 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:11:31.882844 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.882691 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:11:31.882844 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.882726 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:11:31.882844 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.882737 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:11:31.882844 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.882772 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-253.ec2.internal" Jul 08 19:11:31.889949 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.889933 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-253.ec2.internal" Jul 08 19:11:31.890016 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:31.889957 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-253.ec2.internal\": node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:31.895760 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.895735 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 19:11:31.897074 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.897057 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 19:11:31.897168 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.897090 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 19:11:31.897168 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.897113 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 19:11:31.897168 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:31.897157 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 19:11:31.899517 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.899501 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:11:31.904614 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:31.904598 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:31.998256 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.998214 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal"] Jul 08 19:11:31.998411 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.998299 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:11:31.999251 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.999234 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:11:31.999352 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.999268 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:11:31.999352 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:31.999282 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:11:32.000880 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.000864 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:11:32.000994 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.000979 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.001045 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.001009 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:11:32.001643 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.001627 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:11:32.001739 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.001644 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:11:32.001739 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.001654 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:11:32.001739 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.001668 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:11:32.001739 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.001684 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:11:32.001739 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.001669 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:11:32.003212 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.003196 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.003264 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.003222 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:11:32.003998 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.003979 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:11:32.004095 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.004012 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:11:32.004095 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.004027 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:11:32.004993 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:32.004978 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:32.016067 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.016029 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfec9c2f91da29d91a06b5abe6ea341c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal\" (UID: \"cfec9c2f91da29d91a06b5abe6ea341c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.016067 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.016054 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/691ef27feea25903bb266b1bd1efee6e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-253.ec2.internal\" (UID: \"691ef27feea25903bb266b1bd1efee6e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.016158 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.016073 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cfec9c2f91da29d91a06b5abe6ea341c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal\" (UID: \"cfec9c2f91da29d91a06b5abe6ea341c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.034189 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:32.034170 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-253.ec2.internal\" not found" node="ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.039339 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:32.039322 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-253.ec2.internal\" not found" node="ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.105319 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:32.105291 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:32.117018 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.117000 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cfec9c2f91da29d91a06b5abe6ea341c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal\" (UID: \"cfec9c2f91da29d91a06b5abe6ea341c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.117096 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.117022 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfec9c2f91da29d91a06b5abe6ea341c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal\" (UID: \"cfec9c2f91da29d91a06b5abe6ea341c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.117096 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.117043 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/691ef27feea25903bb266b1bd1efee6e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-253.ec2.internal\" (UID: \"691ef27feea25903bb266b1bd1efee6e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.117096 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.117087 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/691ef27feea25903bb266b1bd1efee6e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-253.ec2.internal\" (UID: \"691ef27feea25903bb266b1bd1efee6e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.117211 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.117106 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cfec9c2f91da29d91a06b5abe6ea341c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal\" (UID: \"cfec9c2f91da29d91a06b5abe6ea341c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.117211 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.117125 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfec9c2f91da29d91a06b5abe6ea341c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal\" (UID: \"cfec9c2f91da29d91a06b5abe6ea341c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.205978 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:32.205931 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:32.313311 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:32.313232 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:32.337112 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.337087 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.342304 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.342287 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal" Jul 08 19:11:32.414590 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:32.414547 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:32.518917 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:32.518885 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:32.619268 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:32.619190 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:32.645369 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.645347 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 19:11:32.645546 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.645525 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 19:11:32.645610 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.645544 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 19:11:32.715501 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.715451 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 19:06:31 +0000 UTC" deadline="2028-01-15 15:31:02.308869611 +0000 UTC" Jul 08 19:11:32.715501 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.715494 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13340h19m29.593379596s" Jul 08 19:11:32.719323 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:32.719303 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:32.754995 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.754970 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:11:32.820493 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:32.820464 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:32.921406 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:32.921329 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:32.949902 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:32.949869 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod691ef27feea25903bb266b1bd1efee6e.slice/crio-8f60ea09df6e55f9cc5b684d93f72e704b155497bdb6dff3a3527fbc1e968551 WatchSource:0}: Error finding container 8f60ea09df6e55f9cc5b684d93f72e704b155497bdb6dff3a3527fbc1e968551: Status 404 returned error can't find the container with id 8f60ea09df6e55f9cc5b684d93f72e704b155497bdb6dff3a3527fbc1e968551 Jul 08 19:11:32.950241 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:32.950218 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfec9c2f91da29d91a06b5abe6ea341c.slice/crio-131f4f5473ff14da75d49a89be064a51afdb80ba28baeae36b9e2f39f48cf7be WatchSource:0}: Error finding container 131f4f5473ff14da75d49a89be064a51afdb80ba28baeae36b9e2f39f48cf7be: Status 404 returned error can't find the container with id 131f4f5473ff14da75d49a89be064a51afdb80ba28baeae36b9e2f39f48cf7be Jul 08 19:11:32.955603 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:32.955581 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:11:33.022441 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:33.022407 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found" Jul 08 19:11:33.069100 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.069076 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:11:33.114302 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.114277 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" Jul 08 19:11:33.126307 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.126280 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 19:11:33.127381 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.127359 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal" Jul 08 19:11:33.136065 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.136051 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 19:11:33.497627 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.497599 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:11:33.695161 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.694945 2563 apiserver.go:52] "Watching apiserver" Jul 08 19:11:33.702470 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.702272 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 19:11:33.702685 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.702658 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x","openshift-cluster-node-tuning-operator/tuned-6drtl","openshift-dns/node-resolver-ppkjh","openshift-multus/multus-additional-cni-plugins-kkk4m","openshift-network-diagnostics/network-check-target-xg866","openshift-ovn-kubernetes/ovnkube-node-gcr4l","kube-system/global-pull-secret-syncer-z22h9","openshift-image-registry/node-ca-n2kkx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal","openshift-multus/multus-dqm8n","openshift-multus/network-metrics-daemon-xthzl","openshift-network-operator/iptables-alerter-h8tdr","kube-system/konnectivity-agent-jdh6r"] Jul 08 19:11:33.704212 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.704187 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:33.704329 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:33.704280 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:11:33.705395 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.705360 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h8tdr" Jul 08 19:11:33.707040 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.706854 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.707723 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.707697 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ppkjh" Jul 08 19:11:33.708687 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.708363 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:11:33.708687 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.708367 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 19:11:33.708687 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.708613 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 19:11:33.709514 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.708888 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d6bj5\"" Jul 08 19:11:33.709514 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.709207 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 19:11:33.710031 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.710007 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 19:11:33.711178 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.710205 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 19:11:33.711178 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.710416 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 19:11:33.711178 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.710622 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dl66h\"" Jul 08 19:11:33.711178 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.710765 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 19:11:33.711178 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.711051 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 19:11:33.711502 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.711246 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l9x4w\"" Jul 08 19:11:33.712426 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.712406 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.715175 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.715134 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 19:11:33.715302 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.715262 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:33.715364 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:33.715326 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:11:33.715832 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.715133 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.717108 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.717086 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 19:11:33.717221 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.717205 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 19:11:33.717302 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.717286 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 19:11:33.717444 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.717427 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 19:11:33.717526 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.717511 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c69kn\"" Jul 08 19:11:33.717602 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.717582 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 19:11:33.718070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.718047 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 19:11:33.718181 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.718165 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n2kkx" Jul 08 19:11:33.720577 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.720554 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 19:11:33.720901 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.720884 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 19:11:33.721300 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.721286 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s6xhd\"" Jul 08 19:11:33.726394 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.726346 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/508e47f4-f845-4e04-88dd-1ab13d22d3b8-kubelet-config\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:33.726484 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.726412 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-multus-conf-dir\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.726484 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.726450 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-node-log\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.726589 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.726482 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88c99532-15f1-42d0-9c6e-5e79e3b02932-env-overrides\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.726589 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.726513 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-multus-socket-dir-parent\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.726701 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.726544 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/237823e9-8de2-4851-b96f-ec6cf742d306-tmp-dir\") pod \"node-resolver-ppkjh\" (UID: \"237823e9-8de2-4851-b96f-ec6cf742d306\") " pod="openshift-dns/node-resolver-ppkjh" Jul 08 19:11:33.726754 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.726726 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-log-socket\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.726803 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.726757 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p2zd\" (UniqueName: \"kubernetes.io/projected/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-kube-api-access-4p2zd\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.726803 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.726791 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/82791c49-ac84-485b-8afc-90db19373319-iptables-alerter-script\") pod \"iptables-alerter-h8tdr\" (UID: \"82791c49-ac84-485b-8afc-90db19373319\") " pod="openshift-network-operator/iptables-alerter-h8tdr" Jul 08 19:11:33.726901 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.726819 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l7zv\" (UniqueName: \"kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv\") pod \"network-check-target-xg866\" (UID: \"9d287a39-6145-4cc3-b16f-226c4d738dd9\") " pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:33.726901 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.726853 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-slash\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.726901 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.726887 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-run-netns\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.727044 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.726917 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-run-ovn-kubernetes\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.727044 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.726968 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/508e47f4-f845-4e04-88dd-1ab13d22d3b8-dbus\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:33.727135 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727103 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-run-k8s-cni-cncf-io\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.727221 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727156 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gd7d\" (UniqueName: \"kubernetes.io/projected/237823e9-8de2-4851-b96f-ec6cf742d306-kube-api-access-7gd7d\") pod \"node-resolver-ppkjh\" (UID: \"237823e9-8de2-4851-b96f-ec6cf742d306\") " pod="openshift-dns/node-resolver-ppkjh" Jul 08 19:11:33.727221 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727208 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-kubelet\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.727396 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727339 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-cni-netd\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.727496 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727426 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-cnibin\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.727496 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727485 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-systemd-units\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.727609 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727525 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-var-lib-openvswitch\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.727609 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727575 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88c99532-15f1-42d0-9c6e-5e79e3b02932-ovnkube-config\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.727713 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727609 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-cni-bin\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.727713 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727646 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-var-lib-cni-bin\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.727713 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727671 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-etc-kubernetes\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.727713 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727700 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-run-systemd\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.727898 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727737 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-etc-openvswitch\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.727898 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727772 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-multus-cni-dir\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.727898 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727831 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1205cb15-a154-4f63-857a-fcee1ccaea58-os-release\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.727898 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727871 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:33.728091 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727902 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-run-netns\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.728091 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727934 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/237823e9-8de2-4851-b96f-ec6cf742d306-hosts-file\") pod \"node-resolver-ppkjh\" (UID: \"237823e9-8de2-4851-b96f-ec6cf742d306\") " pod="openshift-dns/node-resolver-ppkjh" Jul 08 19:11:33.728091 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.727989 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88c99532-15f1-42d0-9c6e-5e79e3b02932-ovnkube-script-lib\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.728091 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728032 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6825\" (UniqueName: \"kubernetes.io/projected/88c99532-15f1-42d0-9c6e-5e79e3b02932-kube-api-access-c6825\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.728287 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728094 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-var-lib-cni-multus\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.728287 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728146 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-var-lib-kubelet\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.728287 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728193 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728189 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-multus-daemon-config\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728551 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-run-openvswitch\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728595 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1205cb15-a154-4f63-857a-fcee1ccaea58-cni-binary-copy\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728633 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1205cb15-a154-4f63-857a-fcee1ccaea58-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728673 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xz5x\" (UniqueName: \"kubernetes.io/projected/1205cb15-a154-4f63-857a-fcee1ccaea58-kube-api-access-9xz5x\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728709 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88c99532-15f1-42d0-9c6e-5e79e3b02932-ovn-node-metrics-cert\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728750 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-os-release\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728791 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1205cb15-a154-4f63-857a-fcee1ccaea58-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728834 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728869 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1205cb15-a154-4f63-857a-fcee1ccaea58-system-cni-dir\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728905 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1205cb15-a154-4f63-857a-fcee1ccaea58-cnibin\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728961 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-system-cni-dir\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728998 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-cni-binary-copy\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.729028 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-hostroot\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.729063 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-run-multus-certs\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.730151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.729095 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82791c49-ac84-485b-8afc-90db19373319-host-slash\") pod \"iptables-alerter-h8tdr\" (UID: \"82791c49-ac84-485b-8afc-90db19373319\") " pod="openshift-network-operator/iptables-alerter-h8tdr" Jul 08 19:11:33.731110 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.729130 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whftb\" (UniqueName: \"kubernetes.io/projected/82791c49-ac84-485b-8afc-90db19373319-kube-api-access-whftb\") pod \"iptables-alerter-h8tdr\" (UID: \"82791c49-ac84-485b-8afc-90db19373319\") " pod="openshift-network-operator/iptables-alerter-h8tdr" Jul 08 19:11:33.731110 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.729171 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1205cb15-a154-4f63-857a-fcee1ccaea58-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.731110 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.729178 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nrgpm\"" Jul 08 19:11:33.731110 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.729195 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-run-ovn\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.731110 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.728962 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 19:11:33.731110 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.729179 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 19:11:33.731110 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.729599 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 19:11:33.731110 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.730592 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:33.731110 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:33.730684 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:11:33.732180 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.731199 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2mxsm\"" Jul 08 19:11:33.732180 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.731508 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 19:11:33.732180 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.731818 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:11:33.732180 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.732092 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 19:11:33.733933 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.733486 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.735812 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.735785 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jdh6r" Jul 08 19:11:33.736400 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.736245 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xgz9h\"" Jul 08 19:11:33.736400 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.736252 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 19:11:33.736400 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.736284 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 19:11:33.736400 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.736286 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 19:11:33.738683 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.738666 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 19:11:33.738875 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.738859 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 19:11:33.738971 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.738885 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cxh7t\"" Jul 08 19:11:33.753398 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.753344 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-b9tzv" Jul 08 19:11:33.761043 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.761022 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-b9tzv" Jul 08 19:11:33.814850 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.814822 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 19:11:33.829625 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829594 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1205cb15-a154-4f63-857a-fcee1ccaea58-os-release\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.829771 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829635 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:33.829771 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829667 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z542h\" (UniqueName: \"kubernetes.io/projected/352fc82d-a3e3-4157-ae41-5eade870e4f0-kube-api-access-z542h\") pod \"node-ca-n2kkx\" (UID: \"352fc82d-a3e3-4157-ae41-5eade870e4f0\") " pod="openshift-image-registry/node-ca-n2kkx" Jul 08 19:11:33.829771 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829694 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-run-netns\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.829771 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829718 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/237823e9-8de2-4851-b96f-ec6cf742d306-hosts-file\") pod \"node-resolver-ppkjh\" (UID: \"237823e9-8de2-4851-b96f-ec6cf742d306\") " pod="openshift-dns/node-resolver-ppkjh" Jul 08 19:11:33.829771 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829742 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-host\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.829771 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829767 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88c99532-15f1-42d0-9c6e-5e79e3b02932-ovnkube-script-lib\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.830187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829791 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6825\" (UniqueName: \"kubernetes.io/projected/88c99532-15f1-42d0-9c6e-5e79e3b02932-kube-api-access-c6825\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.830187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829819 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-var-lib-cni-multus\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.830187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829845 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-var-lib-kubelet\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.830187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829869 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-multus-daemon-config\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.830187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829893 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-run-openvswitch\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.830187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829916 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-tuned\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.830187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829940 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1205cb15-a154-4f63-857a-fcee1ccaea58-cni-binary-copy\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.830187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829964 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1205cb15-a154-4f63-857a-fcee1ccaea58-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.830187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.829996 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xz5x\" (UniqueName: \"kubernetes.io/projected/1205cb15-a154-4f63-857a-fcee1ccaea58-kube-api-access-9xz5x\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.830187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830022 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88c99532-15f1-42d0-9c6e-5e79e3b02932-ovn-node-metrics-cert\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.830187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830046 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.830187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830127 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-var-lib-cni-multus\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.830187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830175 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-var-lib-kubelet\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830357 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1205cb15-a154-4f63-857a-fcee1ccaea58-os-release\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:33.830498 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:33.830573 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret podName:508e47f4-f845-4e04-88dd-1ab13d22d3b8 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:34.330551524 +0000 UTC m=+2.999803206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret") pod "global-pull-secret-syncer-z22h9" (UID: "508e47f4-f845-4e04-88dd-1ab13d22d3b8") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830572 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/237823e9-8de2-4851-b96f-ec6cf742d306-hosts-file\") pod \"node-resolver-ppkjh\" (UID: \"237823e9-8de2-4851-b96f-ec6cf742d306\") " pod="openshift-dns/node-resolver-ppkjh" Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830628 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdpmf\" (UniqueName: \"kubernetes.io/projected/f0e112ce-529b-4df8-bc89-ab5879946d5d-kube-api-access-gdpmf\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830648 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-run-netns\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830660 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-os-release\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830687 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1205cb15-a154-4f63-857a-fcee1ccaea58-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830700 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-run-openvswitch\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830714 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830742 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-modprobe-d\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830765 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-sysconfig\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830788 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-multus-daemon-config\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830790 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-sysctl-conf\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.830863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830856 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-os-release\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.831621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830890 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-systemd\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.831621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830917 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs\") pod \"network-metrics-daemon-xthzl\" (UID: \"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2\") " pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:33.831621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830921 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1205cb15-a154-4f63-857a-fcee1ccaea58-cni-binary-copy\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.831621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830945 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.831621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.830946 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1205cb15-a154-4f63-857a-fcee1ccaea58-system-cni-dir\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.831621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831327 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1205cb15-a154-4f63-857a-fcee1ccaea58-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.831621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831393 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1205cb15-a154-4f63-857a-fcee1ccaea58-system-cni-dir\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.831621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831440 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1205cb15-a154-4f63-857a-fcee1ccaea58-cnibin\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.831621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831438 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1205cb15-a154-4f63-857a-fcee1ccaea58-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.831621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831479 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-system-cni-dir\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.831621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831518 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-cni-binary-copy\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.831621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831559 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-hostroot\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.831621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831588 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-run-multus-certs\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.831621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831559 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88c99532-15f1-42d0-9c6e-5e79e3b02932-ovnkube-script-lib\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831642 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1205cb15-a154-4f63-857a-fcee1ccaea58-cnibin\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831696 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-socket-dir\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831723 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0e112ce-529b-4df8-bc89-ab5879946d5d-tmp\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831750 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82791c49-ac84-485b-8afc-90db19373319-host-slash\") pod \"iptables-alerter-h8tdr\" (UID: \"82791c49-ac84-485b-8afc-90db19373319\") " pod="openshift-network-operator/iptables-alerter-h8tdr" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831770 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-system-cni-dir\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831800 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whftb\" (UniqueName: \"kubernetes.io/projected/82791c49-ac84-485b-8afc-90db19373319-kube-api-access-whftb\") pod \"iptables-alerter-h8tdr\" (UID: \"82791c49-ac84-485b-8afc-90db19373319\") " pod="openshift-network-operator/iptables-alerter-h8tdr" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831815 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-run-multus-certs\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831826 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1205cb15-a154-4f63-857a-fcee1ccaea58-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831852 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-run-ovn\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831860 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82791c49-ac84-485b-8afc-90db19373319-host-slash\") pod \"iptables-alerter-h8tdr\" (UID: \"82791c49-ac84-485b-8afc-90db19373319\") " pod="openshift-network-operator/iptables-alerter-h8tdr" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831897 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/508e47f4-f845-4e04-88dd-1ab13d22d3b8-kubelet-config\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831922 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-sysctl-d\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831959 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-lib-modules\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831957 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-cni-binary-copy\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.831983 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-multus-conf-dir\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832050 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-hostroot\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.832348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832056 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/508e47f4-f845-4e04-88dd-1ab13d22d3b8-kubelet-config\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832082 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-node-log\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832098 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-multus-conf-dir\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832108 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88c99532-15f1-42d0-9c6e-5e79e3b02932-env-overrides\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832145 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-run-ovn\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832183 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c64dq\" (UniqueName: \"kubernetes.io/projected/715aef17-2536-45af-9306-5497b26cb26c-kube-api-access-c64dq\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832243 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-multus-socket-dir-parent\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832268 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/237823e9-8de2-4851-b96f-ec6cf742d306-tmp-dir\") pod \"node-resolver-ppkjh\" (UID: \"237823e9-8de2-4851-b96f-ec6cf742d306\") " pod="openshift-dns/node-resolver-ppkjh" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832299 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-node-log\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832355 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-log-socket\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832406 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-log-socket\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832415 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bdd159ef-c93c-4f21-aa89-53fab72c3541-konnectivity-ca\") pod \"konnectivity-agent-jdh6r\" (UID: \"bdd159ef-c93c-4f21-aa89-53fab72c3541\") " pod="kube-system/konnectivity-agent-jdh6r" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832473 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-multus-socket-dir-parent\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832484 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p2zd\" (UniqueName: \"kubernetes.io/projected/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-kube-api-access-4p2zd\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832514 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/82791c49-ac84-485b-8afc-90db19373319-iptables-alerter-script\") pod \"iptables-alerter-h8tdr\" (UID: \"82791c49-ac84-485b-8afc-90db19373319\") " pod="openshift-network-operator/iptables-alerter-h8tdr" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832537 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88c99532-15f1-42d0-9c6e-5e79e3b02932-env-overrides\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832541 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-registration-dir\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.833193 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832583 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1205cb15-a154-4f63-857a-fcee1ccaea58-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-var-lib-kubelet\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832601 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832722 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bdd159ef-c93c-4f21-aa89-53fab72c3541-agent-certs\") pod \"konnectivity-agent-jdh6r\" (UID: \"bdd159ef-c93c-4f21-aa89-53fab72c3541\") " pod="kube-system/konnectivity-agent-jdh6r" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832753 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/237823e9-8de2-4851-b96f-ec6cf742d306-tmp-dir\") pod \"node-resolver-ppkjh\" (UID: \"237823e9-8de2-4851-b96f-ec6cf742d306\") " pod="openshift-dns/node-resolver-ppkjh" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832765 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l7zv\" (UniqueName: \"kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv\") pod \"network-check-target-xg866\" (UID: \"9d287a39-6145-4cc3-b16f-226c4d738dd9\") " pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832802 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-slash\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832819 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-run-netns\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832834 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-run-ovn-kubernetes\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832862 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/508e47f4-f845-4e04-88dd-1ab13d22d3b8-dbus\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832877 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-kubernetes\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832885 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-slash\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832892 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-sys\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832907 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-run-k8s-cni-cncf-io\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832923 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-run-netns\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832977 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gd7d\" (UniqueName: \"kubernetes.io/projected/237823e9-8de2-4851-b96f-ec6cf742d306-kube-api-access-7gd7d\") pod \"node-resolver-ppkjh\" (UID: \"237823e9-8de2-4851-b96f-ec6cf742d306\") " pod="openshift-dns/node-resolver-ppkjh" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.832988 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-run-ovn-kubernetes\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.834116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833025 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-run-k8s-cni-cncf-io\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833030 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/82791c49-ac84-485b-8afc-90db19373319-iptables-alerter-script\") pod \"iptables-alerter-h8tdr\" (UID: \"82791c49-ac84-485b-8afc-90db19373319\") " pod="openshift-network-operator/iptables-alerter-h8tdr" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833021 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-kubelet\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833078 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-kubelet\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833090 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/508e47f4-f845-4e04-88dd-1ab13d22d3b8-dbus\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833122 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-cni-netd\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833171 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/352fc82d-a3e3-4157-ae41-5eade870e4f0-host\") pod \"node-ca-n2kkx\" (UID: \"352fc82d-a3e3-4157-ae41-5eade870e4f0\") " pod="openshift-image-registry/node-ca-n2kkx" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833204 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-cnibin\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833240 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-systemd-units\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833267 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-cnibin\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833290 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-systemd-units\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833348 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-var-lib-openvswitch\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833337 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-cni-netd\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833407 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88c99532-15f1-42d0-9c6e-5e79e3b02932-ovnkube-config\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833419 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-var-lib-openvswitch\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833456 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-sys-fs\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833482 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-run\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.835070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833568 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqb6h\" (UniqueName: \"kubernetes.io/projected/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-kube-api-access-vqb6h\") pod \"network-metrics-daemon-xthzl\" (UID: \"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2\") " pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833630 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-cni-bin\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833669 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-var-lib-cni-bin\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833703 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-etc-kubernetes\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833759 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-host-cni-bin\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833764 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-etc-kubernetes\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833785 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-run-systemd\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833815 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-host-var-lib-cni-bin\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833820 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-etc-openvswitch\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833869 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-run-systemd\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833873 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88c99532-15f1-42d0-9c6e-5e79e3b02932-etc-openvswitch\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833918 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833942 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-device-dir\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.833991 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/352fc82d-a3e3-4157-ae41-5eade870e4f0-serviceca\") pod \"node-ca-n2kkx\" (UID: \"352fc82d-a3e3-4157-ae41-5eade870e4f0\") " pod="openshift-image-registry/node-ca-n2kkx" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.834015 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-multus-cni-dir\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.834119 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-multus-cni-dir\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.836059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.834125 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88c99532-15f1-42d0-9c6e-5e79e3b02932-ovnkube-config\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.838209 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.838166 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88c99532-15f1-42d0-9c6e-5e79e3b02932-ovn-node-metrics-cert\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.842186 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:33.842128 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:11:33.842186 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:33.842147 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:11:33.842336 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:33.842197 2563 projected.go:196] Error preparing data for projected volume kube-api-access-9l7zv for pod openshift-network-diagnostics/network-check-target-xg866: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:11:33.842336 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:33.842252 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv podName:9d287a39-6145-4cc3-b16f-226c4d738dd9 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:34.342233073 +0000 UTC m=+3.011484763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9l7zv" (UniqueName: "kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv") pod "network-check-target-xg866" (UID: "9d287a39-6145-4cc3-b16f-226c4d738dd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:11:33.844843 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.844812 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xz5x\" (UniqueName: \"kubernetes.io/projected/1205cb15-a154-4f63-857a-fcee1ccaea58-kube-api-access-9xz5x\") pod \"multus-additional-cni-plugins-kkk4m\" (UID: \"1205cb15-a154-4f63-857a-fcee1ccaea58\") " pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:33.845022 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.844897 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6825\" (UniqueName: \"kubernetes.io/projected/88c99532-15f1-42d0-9c6e-5e79e3b02932-kube-api-access-c6825\") pod \"ovnkube-node-gcr4l\" (UID: \"88c99532-15f1-42d0-9c6e-5e79e3b02932\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:33.845836 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.845810 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p2zd\" (UniqueName: \"kubernetes.io/projected/ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42-kube-api-access-4p2zd\") pod \"multus-dqm8n\" (UID: \"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42\") " pod="openshift-multus/multus-dqm8n" Jul 08 19:11:33.846772 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.846749 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gd7d\" (UniqueName: \"kubernetes.io/projected/237823e9-8de2-4851-b96f-ec6cf742d306-kube-api-access-7gd7d\") pod \"node-resolver-ppkjh\" (UID: \"237823e9-8de2-4851-b96f-ec6cf742d306\") " pod="openshift-dns/node-resolver-ppkjh" Jul 08 19:11:33.847143 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.847120 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whftb\" (UniqueName: \"kubernetes.io/projected/82791c49-ac84-485b-8afc-90db19373319-kube-api-access-whftb\") pod \"iptables-alerter-h8tdr\" (UID: \"82791c49-ac84-485b-8afc-90db19373319\") " pod="openshift-network-operator/iptables-alerter-h8tdr" Jul 08 19:11:33.902845 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.902790 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal" event={"ID":"691ef27feea25903bb266b1bd1efee6e","Type":"ContainerStarted","Data":"8f60ea09df6e55f9cc5b684d93f72e704b155497bdb6dff3a3527fbc1e968551"} Jul 08 19:11:33.904424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.904359 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" event={"ID":"cfec9c2f91da29d91a06b5abe6ea341c","Type":"ContainerStarted","Data":"131f4f5473ff14da75d49a89be064a51afdb80ba28baeae36b9e2f39f48cf7be"} Jul 08 19:11:33.935072 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935043 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0e112ce-529b-4df8-bc89-ab5879946d5d-tmp\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.935197 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935088 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-sysctl-d\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.935197 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935116 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-lib-modules\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.935197 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935145 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c64dq\" (UniqueName: \"kubernetes.io/projected/715aef17-2536-45af-9306-5497b26cb26c-kube-api-access-c64dq\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.935197 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935173 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bdd159ef-c93c-4f21-aa89-53fab72c3541-konnectivity-ca\") pod \"konnectivity-agent-jdh6r\" (UID: \"bdd159ef-c93c-4f21-aa89-53fab72c3541\") " pod="kube-system/konnectivity-agent-jdh6r" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935201 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-registration-dir\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935225 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-var-lib-kubelet\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935257 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bdd159ef-c93c-4f21-aa89-53fab72c3541-agent-certs\") pod \"konnectivity-agent-jdh6r\" (UID: \"bdd159ef-c93c-4f21-aa89-53fab72c3541\") " pod="kube-system/konnectivity-agent-jdh6r" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935258 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-sysctl-d\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935309 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-kubernetes\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935332 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-sys\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935359 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/352fc82d-a3e3-4157-ae41-5eade870e4f0-host\") pod \"node-ca-n2kkx\" (UID: \"352fc82d-a3e3-4157-ae41-5eade870e4f0\") " pod="openshift-image-registry/node-ca-n2kkx" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935401 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-sys-fs\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935428 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-run\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935452 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqb6h\" (UniqueName: \"kubernetes.io/projected/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-kube-api-access-vqb6h\") pod \"network-metrics-daemon-xthzl\" (UID: \"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2\") " pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935484 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935515 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-device-dir\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935531 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-registration-dir\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935540 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/352fc82d-a3e3-4157-ae41-5eade870e4f0-serviceca\") pod \"node-ca-n2kkx\" (UID: \"352fc82d-a3e3-4157-ae41-5eade870e4f0\") " pod="openshift-image-registry/node-ca-n2kkx" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935607 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z542h\" (UniqueName: \"kubernetes.io/projected/352fc82d-a3e3-4157-ae41-5eade870e4f0-kube-api-access-z542h\") pod \"node-ca-n2kkx\" (UID: \"352fc82d-a3e3-4157-ae41-5eade870e4f0\") " pod="openshift-image-registry/node-ca-n2kkx" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935635 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-host\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.936759 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935673 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-tuned\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935700 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935721 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdpmf\" (UniqueName: \"kubernetes.io/projected/f0e112ce-529b-4df8-bc89-ab5879946d5d-kube-api-access-gdpmf\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935748 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-modprobe-d\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935770 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-sysconfig\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935792 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-sysctl-conf\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935816 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-systemd\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935824 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-var-lib-kubelet\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935839 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs\") pod \"network-metrics-daemon-xthzl\" (UID: \"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2\") " pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935843 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bdd159ef-c93c-4f21-aa89-53fab72c3541-konnectivity-ca\") pod \"konnectivity-agent-jdh6r\" (UID: \"bdd159ef-c93c-4f21-aa89-53fab72c3541\") " pod="kube-system/konnectivity-agent-jdh6r" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935888 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-socket-dir\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:33.935924 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935948 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-lib-modules\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.935966 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/352fc82d-a3e3-4157-ae41-5eade870e4f0-serviceca\") pod \"node-ca-n2kkx\" (UID: \"352fc82d-a3e3-4157-ae41-5eade870e4f0\") " pod="openshift-image-registry/node-ca-n2kkx" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:33.935976 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs podName:fa141408-9f29-44b9-9f55-dcb1fdd5bdf2 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:34.435958112 +0000 UTC m=+3.105209782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs") pod "network-metrics-daemon-xthzl" (UID: "fa141408-9f29-44b9-9f55-dcb1fdd5bdf2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.936019 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-sysconfig\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.936021 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-socket-dir\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.937617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.936064 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-systemd\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.938574 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.936066 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-modprobe-d\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.938574 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.936119 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-kubernetes\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.938574 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.936150 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-sysctl-conf\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.938574 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.936187 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-host\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.938574 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.936240 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-sys-fs\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.938574 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.936252 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.938574 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.936311 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-device-dir\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.938574 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.936337 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-run\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.938574 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.936386 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0e112ce-529b-4df8-bc89-ab5879946d5d-sys\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.938574 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.936400 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/352fc82d-a3e3-4157-ae41-5eade870e4f0-host\") pod \"node-ca-n2kkx\" (UID: \"352fc82d-a3e3-4157-ae41-5eade870e4f0\") " pod="openshift-image-registry/node-ca-n2kkx" Jul 08 19:11:33.938574 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.936425 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/715aef17-2536-45af-9306-5497b26cb26c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.939275 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.939237 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0e112ce-529b-4df8-bc89-ab5879946d5d-tmp\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.940582 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.940556 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bdd159ef-c93c-4f21-aa89-53fab72c3541-agent-certs\") pod \"konnectivity-agent-jdh6r\" (UID: \"bdd159ef-c93c-4f21-aa89-53fab72c3541\") " pod="kube-system/konnectivity-agent-jdh6r" Jul 08 19:11:33.940771 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.940739 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f0e112ce-529b-4df8-bc89-ab5879946d5d-etc-tuned\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.948318 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.948283 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdpmf\" (UniqueName: \"kubernetes.io/projected/f0e112ce-529b-4df8-bc89-ab5879946d5d-kube-api-access-gdpmf\") pod \"tuned-6drtl\" (UID: \"f0e112ce-529b-4df8-bc89-ab5879946d5d\") " pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:33.948472 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.948386 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c64dq\" (UniqueName: \"kubernetes.io/projected/715aef17-2536-45af-9306-5497b26cb26c-kube-api-access-c64dq\") pod \"aws-ebs-csi-driver-node-ljx9x\" (UID: \"715aef17-2536-45af-9306-5497b26cb26c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:33.948711 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.948688 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z542h\" (UniqueName: \"kubernetes.io/projected/352fc82d-a3e3-4157-ae41-5eade870e4f0-kube-api-access-z542h\") pod \"node-ca-n2kkx\" (UID: \"352fc82d-a3e3-4157-ae41-5eade870e4f0\") " pod="openshift-image-registry/node-ca-n2kkx" Jul 08 19:11:33.948974 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:33.948952 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqb6h\" (UniqueName: \"kubernetes.io/projected/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-kube-api-access-vqb6h\") pod \"network-metrics-daemon-xthzl\" (UID: \"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2\") " pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:34.027038 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.026967 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h8tdr" Jul 08 19:11:34.036021 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.035994 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dqm8n" Jul 08 19:11:34.045611 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.045590 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ppkjh" Jul 08 19:11:34.052237 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.052215 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:11:34.060301 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.060279 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kkk4m" Jul 08 19:11:34.067809 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.067794 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n2kkx" Jul 08 19:11:34.077044 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.077023 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6drtl" Jul 08 19:11:34.085911 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.085894 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" Jul 08 19:11:34.093632 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.093614 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jdh6r" Jul 08 19:11:34.235686 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.235646 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:11:34.340608 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.340535 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:34.340746 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:34.340723 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:11:34.340822 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:34.340808 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret podName:508e47f4-f845-4e04-88dd-1ab13d22d3b8 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:35.340786004 +0000 UTC m=+4.010037676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret") pod "global-pull-secret-syncer-z22h9" (UID: "508e47f4-f845-4e04-88dd-1ab13d22d3b8") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:11:34.441861 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.441826 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs\") pod \"network-metrics-daemon-xthzl\" (UID: \"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2\") " pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:34.442017 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.441888 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l7zv\" (UniqueName: \"kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv\") pod \"network-check-target-xg866\" (UID: \"9d287a39-6145-4cc3-b16f-226c4d738dd9\") " pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:34.442082 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:34.442005 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:11:34.442116 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:34.442084 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs podName:fa141408-9f29-44b9-9f55-dcb1fdd5bdf2 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:35.442067491 +0000 UTC m=+4.111319159 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs") pod "network-metrics-daemon-xthzl" (UID: "fa141408-9f29-44b9-9f55-dcb1fdd5bdf2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:11:34.442116 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:34.442016 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:11:34.442186 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:34.442131 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:11:34.442186 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:34.442147 2563 projected.go:196] Error preparing data for projected volume kube-api-access-9l7zv for pod openshift-network-diagnostics/network-check-target-xg866: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:11:34.442255 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:34.442194 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv podName:9d287a39-6145-4cc3-b16f-226c4d738dd9 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:35.442181687 +0000 UTC m=+4.111433370 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9l7zv" (UniqueName: "kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv") pod "network-check-target-xg866" (UID: "9d287a39-6145-4cc3-b16f-226c4d738dd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:11:34.764327 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.764246 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 19:06:33 +0000 UTC" deadline="2027-12-22 07:59:15.407230515 +0000 UTC" Jul 08 19:11:34.764327 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.764281 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12756h47m40.642953482s" Jul 08 19:11:34.898021 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:34.897989 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:34.898163 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:34.898143 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:11:34.994544 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:34.994512 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod715aef17_2536_45af_9306_5497b26cb26c.slice/crio-0e3025677e575565ae12f942118a16992bb9752e1b8f9d2861d8ea3f57566729 WatchSource:0}: Error finding container 0e3025677e575565ae12f942118a16992bb9752e1b8f9d2861d8ea3f57566729: Status 404 returned error can't find the container with id 0e3025677e575565ae12f942118a16992bb9752e1b8f9d2861d8ea3f57566729 Jul 08 19:11:34.994973 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:34.994946 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82791c49_ac84_485b_8afc_90db19373319.slice/crio-7f6f763267373d44547a305206ca06550d23622ac4dee460337bb781d30ab958 WatchSource:0}: Error finding container 7f6f763267373d44547a305206ca06550d23622ac4dee460337bb781d30ab958: Status 404 returned error can't find the container with id 7f6f763267373d44547a305206ca06550d23622ac4dee460337bb781d30ab958 Jul 08 19:11:34.999419 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:11:34.999239 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1205cb15_a154_4f63_857a_fcee1ccaea58.slice/crio-1f0dfe2e43985d6500e80edd9d887b646ae08f3b7237e4f6d4b72e471c693ac3 WatchSource:0}: Error finding container 1f0dfe2e43985d6500e80edd9d887b646ae08f3b7237e4f6d4b72e471c693ac3: Status 404 returned error can't find the container with id 1f0dfe2e43985d6500e80edd9d887b646ae08f3b7237e4f6d4b72e471c693ac3 Jul 08 19:11:35.348940 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.348716 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:35.348940 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:35.348854 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:11:35.349077 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:35.348942 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret podName:508e47f4-f845-4e04-88dd-1ab13d22d3b8 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:37.348928865 +0000 UTC m=+6.018180533 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret") pod "global-pull-secret-syncer-z22h9" (UID: "508e47f4-f845-4e04-88dd-1ab13d22d3b8") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:11:35.449510 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.449475 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs\") pod \"network-metrics-daemon-xthzl\" (UID: \"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2\") " pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:35.449661 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.449525 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l7zv\" (UniqueName: \"kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv\") pod \"network-check-target-xg866\" (UID: \"9d287a39-6145-4cc3-b16f-226c4d738dd9\") " pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:35.449661 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:35.449645 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:11:35.449661 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:35.449646 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:11:35.449834 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:35.449732 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs podName:fa141408-9f29-44b9-9f55-dcb1fdd5bdf2 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:37.449703736 +0000 UTC m=+6.118955410 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs") pod "network-metrics-daemon-xthzl" (UID: "fa141408-9f29-44b9-9f55-dcb1fdd5bdf2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:11:35.449834 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:35.449659 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:11:35.449834 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:35.449750 2563 projected.go:196] Error preparing data for projected volume kube-api-access-9l7zv for pod openshift-network-diagnostics/network-check-target-xg866: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:11:35.449834 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:35.449794 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv podName:9d287a39-6145-4cc3-b16f-226c4d738dd9 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:37.449782517 +0000 UTC m=+6.119034186 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9l7zv" (UniqueName: "kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv") pod "network-check-target-xg866" (UID: "9d287a39-6145-4cc3-b16f-226c4d738dd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:11:35.764843 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.764778 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 19:06:33 +0000 UTC" deadline="2027-12-27 00:09:44.725257102 +0000 UTC" Jul 08 19:11:35.764843 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.764838 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12868h58m8.960423279s" Jul 08 19:11:35.902270 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.900703 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:35.902270 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:35.900830 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:11:35.902270 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.901216 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:35.902270 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:35.901302 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:11:35.919290 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.918582 2563 generic.go:356] "Generic (PLEG): container finished" podID="cfec9c2f91da29d91a06b5abe6ea341c" containerID="ebaa716478ff1cd50fa82b7321ac9a6ece6713dfc8fa0ecf50a8c1a5aa4a413e" exitCode=0 Jul 08 19:11:35.919290 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.918659 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" event={"ID":"cfec9c2f91da29d91a06b5abe6ea341c","Type":"ContainerDied","Data":"ebaa716478ff1cd50fa82b7321ac9a6ece6713dfc8fa0ecf50a8c1a5aa4a413e"} Jul 08 19:11:35.938012 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.937976 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal" event={"ID":"691ef27feea25903bb266b1bd1efee6e","Type":"ContainerStarted","Data":"f4b165385547bd9f37c4f83cd4777aadb245b956e946b144a4a26d1d3486115e"} Jul 08 19:11:35.945961 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.945905 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n2kkx" event={"ID":"352fc82d-a3e3-4157-ae41-5eade870e4f0","Type":"ContainerStarted","Data":"b242eb39c7cb7ce95acd62767e3dc51cc5d07eb59a14b41d9205220dbc5de1fc"} Jul 08 19:11:35.959939 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.959900 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkk4m" event={"ID":"1205cb15-a154-4f63-857a-fcee1ccaea58","Type":"ContainerStarted","Data":"1f0dfe2e43985d6500e80edd9d887b646ae08f3b7237e4f6d4b72e471c693ac3"} Jul 08 19:11:35.975360 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.971476 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h8tdr" event={"ID":"82791c49-ac84-485b-8afc-90db19373319","Type":"ContainerStarted","Data":"7f6f763267373d44547a305206ca06550d23622ac4dee460337bb781d30ab958"} Jul 08 19:11:35.982311 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.982264 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ppkjh" event={"ID":"237823e9-8de2-4851-b96f-ec6cf742d306","Type":"ContainerStarted","Data":"d61fcec44ef0534de37f425ba83ac929fbd21b7f1ae8d75e6ec602e3c6a28bef"} Jul 08 19:11:35.999708 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:35.999632 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dqm8n" event={"ID":"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42","Type":"ContainerStarted","Data":"3768cb848bbca60d4338916bc19c3917242621b3ebe5c5fa0161b5fbe3c140e5"} Jul 08 19:11:36.021484 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:36.021425 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" event={"ID":"88c99532-15f1-42d0-9c6e-5e79e3b02932","Type":"ContainerStarted","Data":"b641938bd93283de72efe93238bd925149338e9f7be55cfa05d6195312cd210d"} Jul 08 19:11:36.035804 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:36.035774 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jdh6r" event={"ID":"bdd159ef-c93c-4f21-aa89-53fab72c3541","Type":"ContainerStarted","Data":"74285ab59b074be6cfef4d38f375a4177cd385e73919fc0921d4eea76bc7be01"} Jul 08 19:11:36.049780 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:36.049747 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" event={"ID":"715aef17-2536-45af-9306-5497b26cb26c","Type":"ContainerStarted","Data":"0e3025677e575565ae12f942118a16992bb9752e1b8f9d2861d8ea3f57566729"} Jul 08 19:11:36.063040 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:36.062972 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6drtl" event={"ID":"f0e112ce-529b-4df8-bc89-ab5879946d5d","Type":"ContainerStarted","Data":"4a2d23da92dcf885e4e53d303cf218b5c71208326e3d6b9ddc5c2933182a3b48"} Jul 08 19:11:36.898752 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:36.898714 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:36.899210 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:36.898853 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:11:37.094409 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:37.094358 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" event={"ID":"cfec9c2f91da29d91a06b5abe6ea341c","Type":"ContainerStarted","Data":"432be6a8e37c81430fa794b7ee96d1ca0051c0e24f983235c14babc2938bbbd8"} Jul 08 19:11:37.112549 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:37.112483 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal" podStartSLOduration=4.112463221 podStartE2EDuration="4.112463221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:35.959850445 +0000 UTC m=+4.629102130" watchObservedRunningTime="2026-07-08 19:11:37.112463221 +0000 UTC m=+5.781714909" Jul 08 19:11:37.366484 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:37.366401 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:37.366650 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:37.366553 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:11:37.366650 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:37.366615 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret podName:508e47f4-f845-4e04-88dd-1ab13d22d3b8 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:41.366596907 +0000 UTC m=+10.035848575 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret") pod "global-pull-secret-syncer-z22h9" (UID: "508e47f4-f845-4e04-88dd-1ab13d22d3b8") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:11:37.466790 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:37.466754 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs\") pod \"network-metrics-daemon-xthzl\" (UID: \"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2\") " pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:37.466952 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:37.466822 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l7zv\" (UniqueName: \"kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv\") pod \"network-check-target-xg866\" (UID: \"9d287a39-6145-4cc3-b16f-226c4d738dd9\") " pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:37.467019 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:37.466973 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:11:37.467019 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:37.466994 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:11:37.467019 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:37.467008 2563 projected.go:196] Error preparing data for projected volume kube-api-access-9l7zv for pod openshift-network-diagnostics/network-check-target-xg866: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:11:37.467171 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:37.467064 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv podName:9d287a39-6145-4cc3-b16f-226c4d738dd9 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:41.467044028 +0000 UTC m=+10.136295698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9l7zv" (UniqueName: "kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv") pod "network-check-target-xg866" (UID: "9d287a39-6145-4cc3-b16f-226c4d738dd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:11:37.467471 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:37.467455 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:11:37.467573 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:37.467504 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs podName:fa141408-9f29-44b9-9f55-dcb1fdd5bdf2 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:41.467489483 +0000 UTC m=+10.136741156 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs") pod "network-metrics-daemon-xthzl" (UID: "fa141408-9f29-44b9-9f55-dcb1fdd5bdf2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:11:37.898925 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:37.898483 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:37.898925 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:37.898483 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:37.898925 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:37.898621 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:11:37.898925 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:37.898652 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:11:38.898050 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:38.898013 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:38.898229 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:38.898179 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:11:39.898301 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:39.898000 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:39.898301 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:39.898131 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:11:39.901153 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:39.901129 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:39.901564 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:39.901284 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:11:40.898112 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:40.897619 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:40.898112 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:40.897770 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:11:41.401173 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:41.401119 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:41.401648 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:41.401275 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:11:41.401648 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:41.401339 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret podName:508e47f4-f845-4e04-88dd-1ab13d22d3b8 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:49.401321456 +0000 UTC m=+18.070573130 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret") pod "global-pull-secret-syncer-z22h9" (UID: "508e47f4-f845-4e04-88dd-1ab13d22d3b8") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:11:41.502137 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:41.502077 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs\") pod \"network-metrics-daemon-xthzl\" (UID: \"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2\") " pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:41.502319 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:41.502151 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l7zv\" (UniqueName: \"kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv\") pod \"network-check-target-xg866\" (UID: \"9d287a39-6145-4cc3-b16f-226c4d738dd9\") " pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:41.502319 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:41.502308 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:11:41.502429 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:41.502330 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:11:41.502429 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:41.502344 2563 projected.go:196] Error preparing data for projected volume kube-api-access-9l7zv for pod openshift-network-diagnostics/network-check-target-xg866: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:11:41.502429 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:41.502421 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv podName:9d287a39-6145-4cc3-b16f-226c4d738dd9 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:49.502401039 +0000 UTC m=+18.171652710 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9l7zv" (UniqueName: "kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv") pod "network-check-target-xg866" (UID: "9d287a39-6145-4cc3-b16f-226c4d738dd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:11:41.502686 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:41.502665 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:11:41.502798 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:41.502729 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs podName:fa141408-9f29-44b9-9f55-dcb1fdd5bdf2 nodeName:}" failed. No retries permitted until 2026-07-08 19:11:49.502713592 +0000 UTC m=+18.171965283 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs") pod "network-metrics-daemon-xthzl" (UID: "fa141408-9f29-44b9-9f55-dcb1fdd5bdf2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:11:41.898802 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:41.898764 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:41.898986 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:41.898886 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:11:41.898986 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:41.898950 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:41.899094 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:41.899056 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:11:42.897792 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:42.897761 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:42.898253 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:42.897910 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:11:43.901107 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:43.901070 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:43.901623 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:43.901202 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:11:43.901623 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:43.901259 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:43.901623 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:43.901341 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:11:44.898118 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:44.898084 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:44.898297 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:44.898217 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:11:45.898074 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:45.898041 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:45.898510 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:45.898142 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:11:45.898510 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:45.898151 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:45.898510 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:45.898265 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:11:46.898203 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:46.898146 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:46.898679 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:46.898300 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:11:47.898335 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:47.898305 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:47.898787 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:47.898303 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:47.898787 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:47.898445 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:11:47.898787 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:47.898483 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:11:48.897948 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:48.897911 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:48.898153 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:48.898056 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:11:49.463576 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:49.463537 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:49.463962 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:49.463677 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:11:49.463962 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:49.463737 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret podName:508e47f4-f845-4e04-88dd-1ab13d22d3b8 nodeName:}" failed. No retries permitted until 2026-07-08 19:12:05.463717491 +0000 UTC m=+34.132969159 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret") pod "global-pull-secret-syncer-z22h9" (UID: "508e47f4-f845-4e04-88dd-1ab13d22d3b8") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:11:49.564135 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:49.564093 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs\") pod \"network-metrics-daemon-xthzl\" (UID: \"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2\") " pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:49.564319 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:49.564160 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l7zv\" (UniqueName: \"kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv\") pod \"network-check-target-xg866\" (UID: \"9d287a39-6145-4cc3-b16f-226c4d738dd9\") " pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:49.564319 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:49.564297 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:11:49.564449 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:49.564330 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:11:49.564449 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:49.564343 2563 projected.go:196] Error preparing data for projected volume kube-api-access-9l7zv for pod openshift-network-diagnostics/network-check-target-xg866: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:11:49.564449 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:49.564299 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:11:49.564449 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:49.564419 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv podName:9d287a39-6145-4cc3-b16f-226c4d738dd9 nodeName:}" failed. No retries permitted until 2026-07-08 19:12:05.564400407 +0000 UTC m=+34.233652090 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9l7zv" (UniqueName: "kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv") pod "network-check-target-xg866" (UID: "9d287a39-6145-4cc3-b16f-226c4d738dd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:11:49.564449 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:49.564448 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs podName:fa141408-9f29-44b9-9f55-dcb1fdd5bdf2 nodeName:}" failed. No retries permitted until 2026-07-08 19:12:05.564434386 +0000 UTC m=+34.233686057 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs") pod "network-metrics-daemon-xthzl" (UID: "fa141408-9f29-44b9-9f55-dcb1fdd5bdf2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:11:49.897884 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:49.897854 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:49.897884 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:49.897875 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:49.898112 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:49.897986 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:11:49.898156 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:49.898121 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:11:50.898388 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:50.898338 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:50.898784 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:50.898495 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:11:51.898124 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:51.898085 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:51.898289 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:51.898176 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:11:51.898289 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:51.898228 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:51.898416 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:51.898315 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:11:52.897564 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:52.897534 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:52.897731 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:52.897645 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:11:53.899525 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:53.899496 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:53.899903 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:53.899615 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:11:53.899903 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:53.899680 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:53.899903 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:53.899748 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:11:54.129697 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.129327 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n2kkx" event={"ID":"352fc82d-a3e3-4157-ae41-5eade870e4f0","Type":"ContainerStarted","Data":"26cbe670cd7b884cd770f60488a9c403dbb0243080d7732305fa1b3e64f6d90c"} Jul 08 19:11:54.130982 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.130958 2563 generic.go:356] "Generic (PLEG): container finished" podID="1205cb15-a154-4f63-857a-fcee1ccaea58" containerID="e45d2ba758907326f5d35447c3ff9933eeac262797b267481e908773f5177363" exitCode=0 Jul 08 19:11:54.131086 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.131027 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkk4m" event={"ID":"1205cb15-a154-4f63-857a-fcee1ccaea58","Type":"ContainerDied","Data":"e45d2ba758907326f5d35447c3ff9933eeac262797b267481e908773f5177363"} Jul 08 19:11:54.132835 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.132807 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ppkjh" event={"ID":"237823e9-8de2-4851-b96f-ec6cf742d306","Type":"ContainerStarted","Data":"245627b47a4ea0f51cb99fa8825b5dccd2ed650cb80e8663d4f0caf08f83b033"} Jul 08 19:11:54.134767 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.134711 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dqm8n" event={"ID":"ce7b6a9f-6b23-4ff0-84bb-15374e4f2e42","Type":"ContainerStarted","Data":"4105306d6d051a27014acb36e15d85700e5131b717e5cc0c56a8946610ffd8bf"} Jul 08 19:11:54.137306 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.137286 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" event={"ID":"88c99532-15f1-42d0-9c6e-5e79e3b02932","Type":"ContainerStarted","Data":"6dafe53c52ed554453aff18b7534d38a2c3b474cc6f127bc4cb5168bfe35c0ed"} Jul 08 19:11:54.137408 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.137340 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" event={"ID":"88c99532-15f1-42d0-9c6e-5e79e3b02932","Type":"ContainerStarted","Data":"6a985fa3578e691b7065a603050b6752d9ee6598b52405dd2f0d22bae376a219"} Jul 08 19:11:54.137408 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.137369 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" event={"ID":"88c99532-15f1-42d0-9c6e-5e79e3b02932","Type":"ContainerStarted","Data":"432114c57d0721aa485c90deb5830928b9b2fb1a5e1baccfcdcb715aa854e37a"} Jul 08 19:11:54.139029 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.139002 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jdh6r" event={"ID":"bdd159ef-c93c-4f21-aa89-53fab72c3541","Type":"ContainerStarted","Data":"67e9dcd699e8012ad8866b863eca346d6439361f2be36ed4e3339bff66af2e82"} Jul 08 19:11:54.140529 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.140506 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" event={"ID":"715aef17-2536-45af-9306-5497b26cb26c","Type":"ContainerStarted","Data":"7ba1851141a2a62574740462f590fe23cc932051332fe9d10983ba9c46b9d8ec"} Jul 08 19:11:54.141778 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.141760 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6drtl" event={"ID":"f0e112ce-529b-4df8-bc89-ab5879946d5d","Type":"ContainerStarted","Data":"22593ff945bd0a1d118db38463b386d784020c0d0a60213a0def16810dffcfaa"} Jul 08 19:11:54.146272 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.146237 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-n2kkx" podStartSLOduration=3.353522489 podStartE2EDuration="22.146225557s" totalImagesPullingTime="18.792703068s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:54.145678443 +0000 UTC m=+22.814930133" watchObservedRunningTime="2026-07-08 19:11:54.146225557 +0000 UTC m=+22.815477244" Jul 08 19:11:54.146387 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.146301 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" podStartSLOduration=21.146295075 podStartE2EDuration="21.146295075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:37.113115279 +0000 UTC m=+5.782366970" watchObservedRunningTime="2026-07-08 19:11:54.146295075 +0000 UTC m=+22.815546762" Jul 08 19:11:54.180643 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.180599 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jdh6r" podStartSLOduration=3.389814003 podStartE2EDuration="22.180583899s" totalImagesPullingTime="18.790769896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:54.163541617 +0000 UTC m=+22.832793308" watchObservedRunningTime="2026-07-08 19:11:54.180583899 +0000 UTC m=+22.849835588" Jul 08 19:11:54.180903 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.180863 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ppkjh" podStartSLOduration=4.377664751 podStartE2EDuration="23.180852816s" totalImagesPullingTime="18.803188065s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:54.18015272 +0000 UTC m=+22.849404403" watchObservedRunningTime="2026-07-08 19:11:54.180852816 +0000 UTC m=+22.850104507" Jul 08 19:11:54.223461 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.223417 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dqm8n" podStartSLOduration=4.362297142 podStartE2EDuration="23.223402373s" totalImagesPullingTime="18.861105231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:54.223165623 +0000 UTC m=+22.892417316" watchObservedRunningTime="2026-07-08 19:11:54.223402373 +0000 UTC m=+22.892654060" Jul 08 19:11:54.242833 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.242795 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6drtl" podStartSLOduration=3.44232688 podStartE2EDuration="22.242780337s" totalImagesPullingTime="18.800453457s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:54.242482151 +0000 UTC m=+22.911733842" watchObservedRunningTime="2026-07-08 19:11:54.242780337 +0000 UTC m=+22.912032026" Jul 08 19:11:54.897758 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:54.897723 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:54.897965 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:54.897873 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:11:55.146954 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:55.146866 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" event={"ID":"88c99532-15f1-42d0-9c6e-5e79e3b02932","Type":"ContainerStarted","Data":"e16c0d3c17c3393afa814fda7dc8573530282c02b88921dc1e12b6ed13ae8c22"} Jul 08 19:11:55.146954 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:55.146938 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" event={"ID":"88c99532-15f1-42d0-9c6e-5e79e3b02932","Type":"ContainerStarted","Data":"053e5d7a66090ed65587290162423a52d80b16c6be4bd47789289ce3bd2d4489"} Jul 08 19:11:55.147783 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:55.146969 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" event={"ID":"88c99532-15f1-42d0-9c6e-5e79e3b02932","Type":"ContainerStarted","Data":"2ab7d265411c53249254aecc746f228b9bc76c1627019d5ffbebc58750ea455e"} Jul 08 19:11:55.629753 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:55.629725 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 19:11:55.791019 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:55.790924 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T19:11:55.629747994Z","UUID":"581005bf-ff06-4f18-a895-29432378d27f","Handler":null,"Name":"","Endpoint":""} Jul 08 19:11:55.792461 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:55.792425 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 19:11:55.792585 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:55.792473 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 19:11:55.897348 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:55.897320 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:55.897538 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:55.897446 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:11:55.897538 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:55.897512 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:55.897651 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:55.897607 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:11:56.151049 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:56.150971 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" event={"ID":"715aef17-2536-45af-9306-5497b26cb26c","Type":"ContainerStarted","Data":"8c0932fbf42eca76cf5977261d5be377f94fe32dbbba3f2963039c7c2f3e4c4a"} Jul 08 19:11:56.152403 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:56.152364 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h8tdr" event={"ID":"82791c49-ac84-485b-8afc-90db19373319","Type":"ContainerStarted","Data":"55ca875eb16c1c379db84fdb489185ed2bcde62b669e7647031270a39df71057"} Jul 08 19:11:56.898436 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:56.898341 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:56.898592 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:56.898494 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:11:57.158326 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:57.158272 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" event={"ID":"88c99532-15f1-42d0-9c6e-5e79e3b02932","Type":"ContainerStarted","Data":"1c3cc4f94bff8499f9a8b1e3b3d4f391681ffc57e7fb0e80cfd876cf29368af9"} Jul 08 19:11:57.160610 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:57.160587 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" event={"ID":"715aef17-2536-45af-9306-5497b26cb26c","Type":"ContainerStarted","Data":"04afb7c8b2549dcfca8a760cf91a0b1bdd7d6605dbdbb2427ce8fdbf1297c469"} Jul 08 19:11:57.180766 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:57.180723 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h8tdr" podStartSLOduration=7.384018388 podStartE2EDuration="26.18070879s" totalImagesPullingTime="18.796690402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:56.168549669 +0000 UTC m=+24.837801360" watchObservedRunningTime="2026-07-08 19:11:57.18070879 +0000 UTC m=+25.849960479" Jul 08 19:11:57.180913 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:57.180876 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljx9x" podStartSLOduration=3.566505997 podStartE2EDuration="25.180868334s" totalImagesPullingTime="21.614362337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:11:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:57.180055551 +0000 UTC m=+25.849307241" watchObservedRunningTime="2026-07-08 19:11:57.180868334 +0000 UTC m=+25.850120026" Jul 08 19:11:57.898052 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:57.898018 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:57.898239 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:57.898018 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:57.898239 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:57.898136 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:11:57.898239 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:57.898189 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:11:58.173251 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:58.173169 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jdh6r" Jul 08 19:11:58.173898 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:58.173876 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jdh6r" Jul 08 19:11:58.676695 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:58.676497 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:11:58.755186 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:58.755153 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:11:58.897834 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:58.897804 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:11:58.898021 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:58.897943 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:11:59.165070 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:59.165037 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jdh6r" Jul 08 19:11:59.165557 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:59.165532 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jdh6r" Jul 08 19:11:59.242282 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:59.242246 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:11:59.898369 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:59.898344 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:11:59.898498 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:59.898477 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:11:59.898562 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:11:59.898533 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:11:59.898661 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:11:59.898641 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:12:00.168499 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:00.168251 2563 generic.go:356] "Generic (PLEG): container finished" podID="1205cb15-a154-4f63-857a-fcee1ccaea58" containerID="15875ff94efc787ae69dbb020192ce0a031714adfa4afb6ae10a99386f54ba51" exitCode=0 Jul 08 19:12:00.168655 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:00.168305 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkk4m" event={"ID":"1205cb15-a154-4f63-857a-fcee1ccaea58","Type":"ContainerDied","Data":"15875ff94efc787ae69dbb020192ce0a031714adfa4afb6ae10a99386f54ba51"} Jul 08 19:12:00.172271 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:00.172239 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" event={"ID":"88c99532-15f1-42d0-9c6e-5e79e3b02932","Type":"ContainerStarted","Data":"6aa6187e9756b7f29df3003baf1fb638c9edeb082c5bb7154e9f25e3b132df67"} Jul 08 19:12:00.216960 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:00.216905 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" podStartSLOduration=9.359227046 podStartE2EDuration="28.216880304s" totalImagesPullingTime="18.857653258s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:00.216128108 +0000 UTC m=+28.885379798" watchObservedRunningTime="2026-07-08 19:12:00.216880304 +0000 UTC m=+28.886131993" Jul 08 19:12:00.640725 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:00.640694 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:00.741109 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:00.741076 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:00.899925 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:00.899800 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:12:00.900146 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:00.899953 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:12:01.174799 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:01.174650 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:12:01.174799 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:01.174731 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:12:01.174799 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:01.174777 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:12:01.190104 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:01.190078 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:12:01.190275 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:01.190218 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:12:01.590288 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:01.589510 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z22h9"] Jul 08 19:12:01.590288 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:01.589674 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:12:01.590288 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:01.589780 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:12:01.590580 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:01.590351 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xthzl"] Jul 08 19:12:01.590580 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:01.590465 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:12:01.590688 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:01.590576 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:12:01.592280 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:01.592253 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xg866"] Jul 08 19:12:01.594049 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:01.592419 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:12:01.594049 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:01.592538 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:12:01.602409 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:01.602368 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:01.660896 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:01.660844 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:02.178094 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:02.178065 2563 generic.go:356] "Generic (PLEG): container finished" podID="1205cb15-a154-4f63-857a-fcee1ccaea58" containerID="a6856f48f7530c3d080a1574508a7d48ca1a28af4f68133ac93fbb2c364e5384" exitCode=0 Jul 08 19:12:02.178260 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:02.178156 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkk4m" event={"ID":"1205cb15-a154-4f63-857a-fcee1ccaea58","Type":"ContainerDied","Data":"a6856f48f7530c3d080a1574508a7d48ca1a28af4f68133ac93fbb2c364e5384"} Jul 08 19:12:02.898242 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:02.898211 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:12:02.898242 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:02.898232 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:12:02.898613 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:02.898331 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:12:02.898613 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:02.898461 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:12:03.182811 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:03.182780 2563 generic.go:356] "Generic (PLEG): container finished" podID="1205cb15-a154-4f63-857a-fcee1ccaea58" containerID="62a9033e90eb779c59342be1717b48121fe4fb464fdebf1996881f6eb7d5f35a" exitCode=0 Jul 08 19:12:03.182964 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:03.182844 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkk4m" event={"ID":"1205cb15-a154-4f63-857a-fcee1ccaea58","Type":"ContainerDied","Data":"62a9033e90eb779c59342be1717b48121fe4fb464fdebf1996881f6eb7d5f35a"} Jul 08 19:12:03.898250 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:03.898215 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:12:03.898661 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:03.898351 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:12:04.020764 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:04.020735 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:04.078895 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:04.078867 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:04.898098 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:04.898064 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:12:04.898279 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:04.898195 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xthzl" podUID="fa141408-9f29-44b9-9f55-dcb1fdd5bdf2" Jul 08 19:12:04.898279 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:04.898241 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:12:04.898682 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:04.898329 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z22h9" podUID="508e47f4-f845-4e04-88dd-1ab13d22d3b8" Jul 08 19:12:05.489088 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:05.489004 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:12:05.489221 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:05.489148 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:12:05.489221 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:05.489207 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret podName:508e47f4-f845-4e04-88dd-1ab13d22d3b8 nodeName:}" failed. No retries permitted until 2026-07-08 19:12:37.489192008 +0000 UTC m=+66.158443676 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret") pod "global-pull-secret-syncer-z22h9" (UID: "508e47f4-f845-4e04-88dd-1ab13d22d3b8") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:12:05.554048 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:05.554011 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:05.589981 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:05.589950 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l7zv\" (UniqueName: \"kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv\") pod \"network-check-target-xg866\" (UID: \"9d287a39-6145-4cc3-b16f-226c4d738dd9\") " pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:12:05.590154 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:05.590011 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs\") pod \"network-metrics-daemon-xthzl\" (UID: \"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2\") " pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:12:05.590154 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:05.590106 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:12:05.590154 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:05.590111 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:12:05.590154 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:05.590127 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:12:05.590154 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:05.590138 2563 projected.go:196] Error preparing data for projected volume kube-api-access-9l7zv for pod openshift-network-diagnostics/network-check-target-xg866: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:12:05.590354 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:05.590165 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs podName:fa141408-9f29-44b9-9f55-dcb1fdd5bdf2 nodeName:}" failed. No retries permitted until 2026-07-08 19:12:37.590148411 +0000 UTC m=+66.259400079 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs") pod "network-metrics-daemon-xthzl" (UID: "fa141408-9f29-44b9-9f55-dcb1fdd5bdf2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:12:05.590354 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:05.590182 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv podName:9d287a39-6145-4cc3-b16f-226c4d738dd9 nodeName:}" failed. No retries permitted until 2026-07-08 19:12:37.590174134 +0000 UTC m=+66.259425802 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9l7zv" (UniqueName: "kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv") pod "network-check-target-xg866" (UID: "9d287a39-6145-4cc3-b16f-226c4d738dd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:12:05.898223 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:05.898195 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:12:05.898430 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:05.898309 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xg866" podUID="9d287a39-6145-4cc3-b16f-226c4d738dd9" Jul 08 19:12:06.658863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.658795 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeReady" Jul 08 19:12:06.658994 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.658942 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 19:12:06.703196 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.703165 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7bqln"] Jul 08 19:12:06.705060 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.705045 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:06.706568 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.706545 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hpgnx"] Jul 08 19:12:06.707818 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.707790 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 19:12:06.707929 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.707863 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 19:12:06.707929 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.707862 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4xh72\"" Jul 08 19:12:06.708040 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.708025 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hpgnx" Jul 08 19:12:06.710705 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.710679 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 19:12:06.710705 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.710679 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 19:12:06.710894 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.710743 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bfkrp\"" Jul 08 19:12:06.710894 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.710688 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 19:12:06.717116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.716679 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7bqln"] Jul 08 19:12:06.718660 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.718637 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hpgnx"] Jul 08 19:12:06.800600 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.800574 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbjfq\" (UniqueName: \"kubernetes.io/projected/51e8638f-de06-407c-9625-327c48e1157d-kube-api-access-bbjfq\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:06.800780 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.800608 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51e8638f-de06-407c-9625-327c48e1157d-tmp-dir\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:06.800780 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.800670 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/51e8638f-de06-407c-9625-327c48e1157d-config-volume\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:06.800780 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.800714 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:06.800780 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.800740 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6w8t\" (UniqueName: \"kubernetes.io/projected/a8262b98-73be-424f-9ac6-5f65172997af-kube-api-access-m6w8t\") pod \"ingress-canary-hpgnx\" (UID: \"a8262b98-73be-424f-9ac6-5f65172997af\") " pod="openshift-ingress-canary/ingress-canary-hpgnx" Jul 08 19:12:06.800780 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.800778 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8262b98-73be-424f-9ac6-5f65172997af-cert\") pod \"ingress-canary-hpgnx\" (UID: \"a8262b98-73be-424f-9ac6-5f65172997af\") " pod="openshift-ingress-canary/ingress-canary-hpgnx" Jul 08 19:12:06.828399 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.828350 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:06.898183 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.898150 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:12:06.898388 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.898167 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:12:06.901108 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.901079 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/51e8638f-de06-407c-9625-327c48e1157d-config-volume\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:06.901611 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.901138 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:06.901611 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.901228 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6w8t\" (UniqueName: \"kubernetes.io/projected/a8262b98-73be-424f-9ac6-5f65172997af-kube-api-access-m6w8t\") pod \"ingress-canary-hpgnx\" (UID: \"a8262b98-73be-424f-9ac6-5f65172997af\") " pod="openshift-ingress-canary/ingress-canary-hpgnx" Jul 08 19:12:06.901611 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.901266 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8262b98-73be-424f-9ac6-5f65172997af-cert\") pod \"ingress-canary-hpgnx\" (UID: \"a8262b98-73be-424f-9ac6-5f65172997af\") " pod="openshift-ingress-canary/ingress-canary-hpgnx" Jul 08 19:12:06.901611 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.901270 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 19:12:06.901611 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.901321 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbjfq\" (UniqueName: \"kubernetes.io/projected/51e8638f-de06-407c-9625-327c48e1157d-kube-api-access-bbjfq\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:06.901611 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.901331 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8qd8q\"" Jul 08 19:12:06.901611 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.901352 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51e8638f-de06-407c-9625-327c48e1157d-tmp-dir\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:06.901611 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.901565 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 19:12:06.901990 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.901677 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51e8638f-de06-407c-9625-327c48e1157d-tmp-dir\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:06.901990 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:06.901731 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:12:06.901990 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.901751 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/51e8638f-de06-407c-9625-327c48e1157d-config-volume\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:06.901990 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:06.901788 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls podName:51e8638f-de06-407c-9625-327c48e1157d nodeName:}" failed. No retries permitted until 2026-07-08 19:12:07.401770258 +0000 UTC m=+36.071021925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls") pod "dns-default-7bqln" (UID: "51e8638f-de06-407c-9625-327c48e1157d") : secret "dns-default-metrics-tls" not found Jul 08 19:12:06.906719 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.906698 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8262b98-73be-424f-9ac6-5f65172997af-cert\") pod \"ingress-canary-hpgnx\" (UID: \"a8262b98-73be-424f-9ac6-5f65172997af\") " pod="openshift-ingress-canary/ingress-canary-hpgnx" Jul 08 19:12:06.910494 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.910446 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbjfq\" (UniqueName: \"kubernetes.io/projected/51e8638f-de06-407c-9625-327c48e1157d-kube-api-access-bbjfq\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:06.910770 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:06.910743 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6w8t\" (UniqueName: \"kubernetes.io/projected/a8262b98-73be-424f-9ac6-5f65172997af-kube-api-access-m6w8t\") pod \"ingress-canary-hpgnx\" (UID: \"a8262b98-73be-424f-9ac6-5f65172997af\") " pod="openshift-ingress-canary/ingress-canary-hpgnx" Jul 08 19:12:07.024766 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:07.024730 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hpgnx" Jul 08 19:12:07.176917 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:07.176688 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hpgnx"] Jul 08 19:12:07.181120 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:07.181090 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8262b98_73be_424f_9ac6_5f65172997af.slice/crio-a0bba9010982612db2f259e3036d8fad4407257c189ef760a4643f018f19d7a5 WatchSource:0}: Error finding container a0bba9010982612db2f259e3036d8fad4407257c189ef760a4643f018f19d7a5: Status 404 returned error can't find the container with id a0bba9010982612db2f259e3036d8fad4407257c189ef760a4643f018f19d7a5 Jul 08 19:12:07.193086 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:07.193052 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hpgnx" event={"ID":"a8262b98-73be-424f-9ac6-5f65172997af","Type":"ContainerStarted","Data":"a0bba9010982612db2f259e3036d8fad4407257c189ef760a4643f018f19d7a5"} Jul 08 19:12:07.406475 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:07.406440 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:07.406625 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:07.406598 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:12:07.406696 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:07.406677 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls podName:51e8638f-de06-407c-9625-327c48e1157d nodeName:}" failed. No retries permitted until 2026-07-08 19:12:08.406660463 +0000 UTC m=+37.075912151 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls") pod "dns-default-7bqln" (UID: "51e8638f-de06-407c-9625-327c48e1157d") : secret "dns-default-metrics-tls" not found Jul 08 19:12:07.897442 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:07.897406 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:12:07.900907 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:07.900674 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-92l7d\"" Jul 08 19:12:07.900907 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:07.900694 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 19:12:07.900907 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:07.900750 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 19:12:08.415342 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:08.415299 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:08.415526 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:08.415480 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:12:08.415610 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:08.415549 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls podName:51e8638f-de06-407c-9625-327c48e1157d nodeName:}" failed. No retries permitted until 2026-07-08 19:12:10.415529653 +0000 UTC m=+39.084781344 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls") pod "dns-default-7bqln" (UID: "51e8638f-de06-407c-9625-327c48e1157d") : secret "dns-default-metrics-tls" not found Jul 08 19:12:08.646841 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:08.646796 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:08.747407 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:08.747343 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:09.201174 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:09.201138 2563 generic.go:356] "Generic (PLEG): container finished" podID="1205cb15-a154-4f63-857a-fcee1ccaea58" containerID="47227288fdb46da38d015ef69a59da99a1a949015356917863f31405eeae1403" exitCode=0 Jul 08 19:12:09.201636 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:09.201220 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkk4m" event={"ID":"1205cb15-a154-4f63-857a-fcee1ccaea58","Type":"ContainerDied","Data":"47227288fdb46da38d015ef69a59da99a1a949015356917863f31405eeae1403"} Jul 08 19:12:10.207333 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:10.207302 2563 generic.go:356] "Generic (PLEG): container finished" podID="1205cb15-a154-4f63-857a-fcee1ccaea58" containerID="419cbd8572a7375a87927cbac96083658c8b70d894355e20cf7dd7c76f832530" exitCode=0 Jul 08 19:12:10.207899 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:10.207363 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkk4m" event={"ID":"1205cb15-a154-4f63-857a-fcee1ccaea58","Type":"ContainerDied","Data":"419cbd8572a7375a87927cbac96083658c8b70d894355e20cf7dd7c76f832530"} Jul 08 19:12:10.431776 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:10.431734 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:10.431952 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:10.431899 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:12:10.432008 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:10.431997 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls podName:51e8638f-de06-407c-9625-327c48e1157d nodeName:}" failed. No retries permitted until 2026-07-08 19:12:14.431973729 +0000 UTC m=+43.101225420 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls") pod "dns-default-7bqln" (UID: "51e8638f-de06-407c-9625-327c48e1157d") : secret "dns-default-metrics-tls" not found Jul 08 19:12:10.885183 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:10.885148 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:10.982034 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:10.982005 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:11.210911 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:11.210876 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hpgnx" event={"ID":"a8262b98-73be-424f-9ac6-5f65172997af","Type":"ContainerStarted","Data":"e73f7f022811781327be75e3aa4d6c8cda07ae85df88e05416d1a86e69e63368"} Jul 08 19:12:11.214046 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:11.214019 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkk4m" event={"ID":"1205cb15-a154-4f63-857a-fcee1ccaea58","Type":"ContainerStarted","Data":"be75cb9a5683e076c534085e463685a61e7de55257ddca3ba0c8c50b0b0f817c"} Jul 08 19:12:11.227182 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:11.227139 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hpgnx" podStartSLOduration=1.503522346 podStartE2EDuration="5.227123438s" totalImagesPullingTime="3.723601092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:11.226545897 +0000 UTC m=+39.895797600" watchObservedRunningTime="2026-07-08 19:12:11.227123438 +0000 UTC m=+39.896375106" Jul 08 19:12:11.270237 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:11.270187 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kkk4m" podStartSLOduration=7.106449822 podStartE2EDuration="39.270171113s" totalImagesPullingTime="32.163721291s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:11:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:11.268918354 +0000 UTC m=+39.938170038" watchObservedRunningTime="2026-07-08 19:12:11.270171113 +0000 UTC m=+39.939422844" Jul 08 19:12:11.401873 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:11.401789 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:11.502995 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:11.502962 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:11.676547 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:11.676475 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:11.734865 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:11.734834 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:14.458980 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:14.458938 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:14.459396 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:14.459098 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:12:14.459396 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:14.459169 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls podName:51e8638f-de06-407c-9625-327c48e1157d nodeName:}" failed. No retries permitted until 2026-07-08 19:12:22.459150832 +0000 UTC m=+51.128402523 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls") pod "dns-default-7bqln" (UID: "51e8638f-de06-407c-9625-327c48e1157d") : secret "dns-default-metrics-tls" not found Jul 08 19:12:14.688735 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:14.688693 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:14.788610 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:14.788577 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:15.513834 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.513797 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds"] Jul 08 19:12:15.543130 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.543104 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-rgz68"] Jul 08 19:12:15.543293 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.543234 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" Jul 08 19:12:15.545985 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.545964 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 08 19:12:15.546420 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.546247 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 08 19:12:15.546420 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.546284 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-smrqj\"" Jul 08 19:12:15.546420 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.546300 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 08 19:12:15.546420 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.546302 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:12:15.557098 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.557071 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9"] Jul 08 19:12:15.557205 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.557190 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.560583 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.560564 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 19:12:15.560688 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.560623 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 08 19:12:15.560688 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.560638 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 19:12:15.560688 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.560680 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mnx4x\"" Jul 08 19:12:15.560813 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.560716 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 08 19:12:15.566570 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.566554 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 08 19:12:15.567898 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.567880 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgrr9\" (UniqueName: \"kubernetes.io/projected/1538d1d4-a106-4aa3-9caa-153393cfe60c-kube-api-access-dgrr9\") pod \"service-ca-operator-5d46d9ffd4-2j8ds\" (UID: \"1538d1d4-a106-4aa3-9caa-153393cfe60c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" Jul 08 19:12:15.567956 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.567908 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0e7870b-512b-4301-8ebc-3479c3ac23d3-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.567956 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.567924 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0e7870b-512b-4301-8ebc-3479c3ac23d3-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.567956 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.567951 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1538d1d4-a106-4aa3-9caa-153393cfe60c-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-2j8ds\" (UID: \"1538d1d4-a106-4aa3-9caa-153393cfe60c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" Jul 08 19:12:15.568079 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.567975 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0e7870b-512b-4301-8ebc-3479c3ac23d3-tmp\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.568079 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.568042 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c0e7870b-512b-4301-8ebc-3479c3ac23d3-snapshots\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.568151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.568093 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1538d1d4-a106-4aa3-9caa-153393cfe60c-config\") pod \"service-ca-operator-5d46d9ffd4-2j8ds\" (UID: \"1538d1d4-a106-4aa3-9caa-153393cfe60c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" Jul 08 19:12:15.568151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.568110 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c0e7870b-512b-4301-8ebc-3479c3ac23d3-serving-cert\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.568151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.568133 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkznf\" (UniqueName: \"kubernetes.io/projected/c0e7870b-512b-4301-8ebc-3479c3ac23d3-kube-api-access-vkznf\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.579155 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.579136 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2"] Jul 08 19:12:15.579277 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.579264 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:15.583340 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.583219 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-8vhgk\"" Jul 08 19:12:15.583340 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.583298 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 19:12:15.585552 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.583648 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 08 19:12:15.585552 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.583769 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 19:12:15.585552 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.584873 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 08 19:12:15.599010 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.598989 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds"] Jul 08 19:12:15.599096 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.599019 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2"] Jul 08 19:12:15.599096 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.599036 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-rgz68"] Jul 08 19:12:15.599096 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.599057 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9"] Jul 08 19:12:15.599185 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.599110 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" Jul 08 19:12:15.602018 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.601997 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 08 19:12:15.602114 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.602064 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 08 19:12:15.602114 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.602087 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-dz8c6\"" Jul 08 19:12:15.602291 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.602271 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:12:15.602394 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.602317 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 08 19:12:15.618723 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.618696 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-tppst"] Jul 08 19:12:15.635283 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.635258 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:15.636657 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.636604 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-tppst"] Jul 08 19:12:15.638165 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.638141 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 08 19:12:15.638431 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.638413 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:12:15.638505 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.638473 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-lpr6c\"" Jul 08 19:12:15.639190 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.639170 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 08 19:12:15.639337 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.639174 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 08 19:12:15.644979 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.644961 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 08 19:12:15.668889 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.668868 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1538d1d4-a106-4aa3-9caa-153393cfe60c-config\") pod \"service-ca-operator-5d46d9ffd4-2j8ds\" (UID: \"1538d1d4-a106-4aa3-9caa-153393cfe60c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" Jul 08 19:12:15.669010 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.668895 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c0e7870b-512b-4301-8ebc-3479c3ac23d3-serving-cert\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.669010 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.668922 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e1b57420-0780-41af-8d72-56425fe6a28d-config\") pod \"console-operator-68b757fb67-tppst\" (UID: \"e1b57420-0780-41af-8d72-56425fe6a28d\") " pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:15.669010 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.668938 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e1b57420-0780-41af-8d72-56425fe6a28d-trusted-ca\") pod \"console-operator-68b757fb67-tppst\" (UID: \"e1b57420-0780-41af-8d72-56425fe6a28d\") " pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:15.669010 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.668960 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:15.669010 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.668990 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkznf\" (UniqueName: \"kubernetes.io/projected/c0e7870b-512b-4301-8ebc-3479c3ac23d3-kube-api-access-vkznf\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.669197 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669050 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b42ce879-9228-489a-a7a8-d12970b3cd3a-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-jmjx2\" (UID: \"b42ce879-9228-489a-a7a8-d12970b3cd3a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" Jul 08 19:12:15.669197 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669086 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e1b57420-0780-41af-8d72-56425fe6a28d-serving-cert\") pod \"console-operator-68b757fb67-tppst\" (UID: \"e1b57420-0780-41af-8d72-56425fe6a28d\") " pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:15.669197 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669134 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fq26\" (UniqueName: \"kubernetes.io/projected/b42ce879-9228-489a-a7a8-d12970b3cd3a-kube-api-access-2fq26\") pod \"kube-storage-version-migrator-operator-846bf668f7-jmjx2\" (UID: \"b42ce879-9228-489a-a7a8-d12970b3cd3a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" Jul 08 19:12:15.669197 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669189 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgrr9\" (UniqueName: \"kubernetes.io/projected/1538d1d4-a106-4aa3-9caa-153393cfe60c-kube-api-access-dgrr9\") pod \"service-ca-operator-5d46d9ffd4-2j8ds\" (UID: \"1538d1d4-a106-4aa3-9caa-153393cfe60c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" Jul 08 19:12:15.669403 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669222 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0e7870b-512b-4301-8ebc-3479c3ac23d3-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.669403 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669248 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0e7870b-512b-4301-8ebc-3479c3ac23d3-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.669403 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669277 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cfs4\" (UniqueName: \"kubernetes.io/projected/e1b57420-0780-41af-8d72-56425fe6a28d-kube-api-access-9cfs4\") pod \"console-operator-68b757fb67-tppst\" (UID: \"e1b57420-0780-41af-8d72-56425fe6a28d\") " pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:15.669403 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669314 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1538d1d4-a106-4aa3-9caa-153393cfe60c-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-2j8ds\" (UID: \"1538d1d4-a106-4aa3-9caa-153393cfe60c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" Jul 08 19:12:15.669403 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669343 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0e7870b-512b-4301-8ebc-3479c3ac23d3-tmp\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.669632 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669407 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b42ce879-9228-489a-a7a8-d12970b3cd3a-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-jmjx2\" (UID: \"b42ce879-9228-489a-a7a8-d12970b3cd3a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" Jul 08 19:12:15.669632 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669445 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c0e7870b-512b-4301-8ebc-3479c3ac23d3-snapshots\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.669632 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669574 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1538d1d4-a106-4aa3-9caa-153393cfe60c-config\") pod \"service-ca-operator-5d46d9ffd4-2j8ds\" (UID: \"1538d1d4-a106-4aa3-9caa-153393cfe60c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" Jul 08 19:12:15.669749 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669653 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:15.669749 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669684 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9vtj\" (UniqueName: \"kubernetes.io/projected/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-kube-api-access-k9vtj\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:15.669917 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669896 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0e7870b-512b-4301-8ebc-3479c3ac23d3-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.669987 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.669969 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0e7870b-512b-4301-8ebc-3479c3ac23d3-tmp\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.670038 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.670015 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c0e7870b-512b-4301-8ebc-3479c3ac23d3-snapshots\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.670179 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.670157 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0e7870b-512b-4301-8ebc-3479c3ac23d3-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.672359 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.672341 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1538d1d4-a106-4aa3-9caa-153393cfe60c-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-2j8ds\" (UID: \"1538d1d4-a106-4aa3-9caa-153393cfe60c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" Jul 08 19:12:15.672439 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.672360 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c0e7870b-512b-4301-8ebc-3479c3ac23d3-serving-cert\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.676212 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.676194 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgrr9\" (UniqueName: \"kubernetes.io/projected/1538d1d4-a106-4aa3-9caa-153393cfe60c-kube-api-access-dgrr9\") pod \"service-ca-operator-5d46d9ffd4-2j8ds\" (UID: \"1538d1d4-a106-4aa3-9caa-153393cfe60c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" Jul 08 19:12:15.676479 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.676459 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkznf\" (UniqueName: \"kubernetes.io/projected/c0e7870b-512b-4301-8ebc-3479c3ac23d3-kube-api-access-vkznf\") pod \"insights-operator-5c847dcfc4-rgz68\" (UID: \"c0e7870b-512b-4301-8ebc-3479c3ac23d3\") " pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.712429 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.712400 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-btcbl"] Jul 08 19:12:15.746470 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.746437 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-f8fc8f87d-p5vqg"] Jul 08 19:12:15.746625 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.746523 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-btcbl" Jul 08 19:12:15.749181 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.749158 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jjd28\"" Jul 08 19:12:15.760660 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.760641 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-btcbl"] Jul 08 19:12:15.760729 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.760667 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f8fc8f87d-p5vqg"] Jul 08 19:12:15.760762 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.760731 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.763482 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.763461 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 19:12:15.763582 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.763499 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 19:12:15.763582 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.763461 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 19:12:15.763680 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.763502 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9x74r\"" Jul 08 19:12:15.764063 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.763987 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:15.769198 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.769180 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 19:12:15.770063 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770043 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e1b57420-0780-41af-8d72-56425fe6a28d-trusted-ca\") pod \"console-operator-68b757fb67-tppst\" (UID: \"e1b57420-0780-41af-8d72-56425fe6a28d\") " pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:15.770136 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770070 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-certificates\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.770136 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770097 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flwxk\" (UniqueName: \"kubernetes.io/projected/83db0d91-71be-4616-bbfb-02c836423f7b-kube-api-access-flwxk\") pod \"network-check-source-989f8b8dc-btcbl\" (UID: \"83db0d91-71be-4616-bbfb-02c836423f7b\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-btcbl" Jul 08 19:12:15.770136 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770121 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9vtj\" (UniqueName: \"kubernetes.io/projected/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-kube-api-access-k9vtj\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:15.770237 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770212 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-image-registry-private-configuration\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.770300 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770288 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:15.770332 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770321 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-bound-sa-token\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.770366 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770341 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e1b57420-0780-41af-8d72-56425fe6a28d-serving-cert\") pod \"console-operator-68b757fb67-tppst\" (UID: \"e1b57420-0780-41af-8d72-56425fe6a28d\") " pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:15.770429 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770368 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-installation-pull-secrets\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.770468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770428 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9dfb\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-kube-api-access-m9dfb\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.770468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770456 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cfs4\" (UniqueName: \"kubernetes.io/projected/e1b57420-0780-41af-8d72-56425fe6a28d-kube-api-access-9cfs4\") pod \"console-operator-68b757fb67-tppst\" (UID: \"e1b57420-0780-41af-8d72-56425fe6a28d\") " pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:15.770534 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770477 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b42ce879-9228-489a-a7a8-d12970b3cd3a-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-jmjx2\" (UID: \"b42ce879-9228-489a-a7a8-d12970b3cd3a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" Jul 08 19:12:15.770534 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770495 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:15.770534 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770517 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e1b57420-0780-41af-8d72-56425fe6a28d-config\") pod \"console-operator-68b757fb67-tppst\" (UID: \"e1b57420-0780-41af-8d72-56425fe6a28d\") " pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:15.770534 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770532 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-ca-trust-extracted\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.770714 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770556 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.770714 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770598 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b42ce879-9228-489a-a7a8-d12970b3cd3a-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-jmjx2\" (UID: \"b42ce879-9228-489a-a7a8-d12970b3cd3a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" Jul 08 19:12:15.770714 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:15.770625 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:12:15.770714 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770640 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fq26\" (UniqueName: \"kubernetes.io/projected/b42ce879-9228-489a-a7a8-d12970b3cd3a-kube-api-access-2fq26\") pod \"kube-storage-version-migrator-operator-846bf668f7-jmjx2\" (UID: \"b42ce879-9228-489a-a7a8-d12970b3cd3a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" Jul 08 19:12:15.770714 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.770686 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-trusted-ca\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.771175 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:15.771154 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls podName:9e82e9f4-ccde-4b94-91f9-c5ed181ae83c nodeName:}" failed. No retries permitted until 2026-07-08 19:12:16.271130635 +0000 UTC m=+44.940382318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-w9db9" (UID: "9e82e9f4-ccde-4b94-91f9-c5ed181ae83c") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:12:15.771288 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.771272 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e1b57420-0780-41af-8d72-56425fe6a28d-trusted-ca\") pod \"console-operator-68b757fb67-tppst\" (UID: \"e1b57420-0780-41af-8d72-56425fe6a28d\") " pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:15.771354 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.771342 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b42ce879-9228-489a-a7a8-d12970b3cd3a-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-jmjx2\" (UID: \"b42ce879-9228-489a-a7a8-d12970b3cd3a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" Jul 08 19:12:15.771460 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.771434 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:15.771838 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.771813 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e1b57420-0780-41af-8d72-56425fe6a28d-config\") pod \"console-operator-68b757fb67-tppst\" (UID: \"e1b57420-0780-41af-8d72-56425fe6a28d\") " pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:15.774838 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.773898 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b42ce879-9228-489a-a7a8-d12970b3cd3a-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-jmjx2\" (UID: \"b42ce879-9228-489a-a7a8-d12970b3cd3a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" Jul 08 19:12:15.774838 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.773974 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e1b57420-0780-41af-8d72-56425fe6a28d-serving-cert\") pod \"console-operator-68b757fb67-tppst\" (UID: \"e1b57420-0780-41af-8d72-56425fe6a28d\") " pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:15.777741 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.777717 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9vtj\" (UniqueName: \"kubernetes.io/projected/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-kube-api-access-k9vtj\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:15.781314 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.781294 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cfs4\" (UniqueName: \"kubernetes.io/projected/e1b57420-0780-41af-8d72-56425fe6a28d-kube-api-access-9cfs4\") pod \"console-operator-68b757fb67-tppst\" (UID: \"e1b57420-0780-41af-8d72-56425fe6a28d\") " pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:15.781966 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.781949 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fq26\" (UniqueName: \"kubernetes.io/projected/b42ce879-9228-489a-a7a8-d12970b3cd3a-kube-api-access-2fq26\") pod \"kube-storage-version-migrator-operator-846bf668f7-jmjx2\" (UID: \"b42ce879-9228-489a-a7a8-d12970b3cd3a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" Jul 08 19:12:15.853238 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.853200 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" Jul 08 19:12:15.864697 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.864664 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:15.866426 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.866408 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" Jul 08 19:12:15.871692 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.871672 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-image-registry-private-configuration\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.871750 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.871706 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-bound-sa-token\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.871750 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.871730 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-installation-pull-secrets\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.871877 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.871856 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9dfb\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-kube-api-access-m9dfb\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.871952 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.871924 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-ca-trust-extracted\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.872007 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.871959 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.872067 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.872010 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-trusted-ca\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.872122 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.872065 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-certificates\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.872122 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.872109 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flwxk\" (UniqueName: \"kubernetes.io/projected/83db0d91-71be-4616-bbfb-02c836423f7b-kube-api-access-flwxk\") pod \"network-check-source-989f8b8dc-btcbl\" (UID: \"83db0d91-71be-4616-bbfb-02c836423f7b\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-btcbl" Jul 08 19:12:15.872230 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:15.872149 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:12:15.872230 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:15.872169 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f8fc8f87d-p5vqg: secret "image-registry-tls" not found Jul 08 19:12:15.872312 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:15.872234 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls podName:f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7 nodeName:}" failed. No retries permitted until 2026-07-08 19:12:16.372214138 +0000 UTC m=+45.041465819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls") pod "image-registry-f8fc8f87d-p5vqg" (UID: "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7") : secret "image-registry-tls" not found Jul 08 19:12:15.872312 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.872259 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-ca-trust-extracted\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.872661 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.872637 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-certificates\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.872979 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.872958 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-trusted-ca\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.874123 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.874104 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-installation-pull-secrets\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.874189 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.874153 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-image-registry-private-configuration\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.882761 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.882741 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flwxk\" (UniqueName: \"kubernetes.io/projected/83db0d91-71be-4616-bbfb-02c836423f7b-kube-api-access-flwxk\") pod \"network-check-source-989f8b8dc-btcbl\" (UID: \"83db0d91-71be-4616-bbfb-02c836423f7b\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-btcbl" Jul 08 19:12:15.882849 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.882785 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9dfb\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-kube-api-access-m9dfb\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.882992 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.882975 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-bound-sa-token\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:15.906794 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.906773 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" Jul 08 19:12:15.945129 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:15.944967 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:16.014413 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:16.014277 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds"] Jul 08 19:12:16.028852 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:16.026261 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1538d1d4_a106_4aa3_9caa_153393cfe60c.slice/crio-94a6a28d4d5c11dcdeb254ef8d30ce749907b9ce92d8f29c43969623632c3ae2 WatchSource:0}: Error finding container 94a6a28d4d5c11dcdeb254ef8d30ce749907b9ce92d8f29c43969623632c3ae2: Status 404 returned error can't find the container with id 94a6a28d4d5c11dcdeb254ef8d30ce749907b9ce92d8f29c43969623632c3ae2 Jul 08 19:12:16.055513 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:16.055446 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-btcbl" Jul 08 19:12:16.067341 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:16.067312 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-rgz68"] Jul 08 19:12:16.069603 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:16.069567 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0e7870b_512b_4301_8ebc_3479c3ac23d3.slice/crio-69c332ba5b8968443b850c24f4feb2b207fe0cf543e95326f5fa3c27a4677e9b WatchSource:0}: Error finding container 69c332ba5b8968443b850c24f4feb2b207fe0cf543e95326f5fa3c27a4677e9b: Status 404 returned error can't find the container with id 69c332ba5b8968443b850c24f4feb2b207fe0cf543e95326f5fa3c27a4677e9b Jul 08 19:12:16.120450 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:16.120415 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-tppst"] Jul 08 19:12:16.123544 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:16.123506 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1b57420_0780_41af_8d72_56425fe6a28d.slice/crio-f830367077263d5ba3af33ed4f2b788611c8c8bb5e29dc0a79109645e6cf2b03 WatchSource:0}: Error finding container f830367077263d5ba3af33ed4f2b788611c8c8bb5e29dc0a79109645e6cf2b03: Status 404 returned error can't find the container with id f830367077263d5ba3af33ed4f2b788611c8c8bb5e29dc0a79109645e6cf2b03 Jul 08 19:12:16.134494 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:16.134472 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2"] Jul 08 19:12:16.214336 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:16.214298 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-btcbl"] Jul 08 19:12:16.216931 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:16.216894 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83db0d91_71be_4616_bbfb_02c836423f7b.slice/crio-27cd4fa6f42ac51bcd1f8aabd5472fcdffaabde54369a9774ed57ab6faf3aca9 WatchSource:0}: Error finding container 27cd4fa6f42ac51bcd1f8aabd5472fcdffaabde54369a9774ed57ab6faf3aca9: Status 404 returned error can't find the container with id 27cd4fa6f42ac51bcd1f8aabd5472fcdffaabde54369a9774ed57ab6faf3aca9 Jul 08 19:12:16.226248 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:16.226220 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" event={"ID":"1538d1d4-a106-4aa3-9caa-153393cfe60c","Type":"ContainerStarted","Data":"94a6a28d4d5c11dcdeb254ef8d30ce749907b9ce92d8f29c43969623632c3ae2"} Jul 08 19:12:16.227308 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:16.227286 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tppst" event={"ID":"e1b57420-0780-41af-8d72-56425fe6a28d","Type":"ContainerStarted","Data":"f830367077263d5ba3af33ed4f2b788611c8c8bb5e29dc0a79109645e6cf2b03"} Jul 08 19:12:16.228430 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:16.228400 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" event={"ID":"b42ce879-9228-489a-a7a8-d12970b3cd3a","Type":"ContainerStarted","Data":"edb215f305a0e91a2a1f85bb4058080156faea62d02737382e4ed5268a8ed8af"} Jul 08 19:12:16.229423 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:16.229400 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" event={"ID":"c0e7870b-512b-4301-8ebc-3479c3ac23d3","Type":"ContainerStarted","Data":"69c332ba5b8968443b850c24f4feb2b207fe0cf543e95326f5fa3c27a4677e9b"} Jul 08 19:12:16.230482 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:16.230464 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-btcbl" event={"ID":"83db0d91-71be-4616-bbfb-02c836423f7b","Type":"ContainerStarted","Data":"27cd4fa6f42ac51bcd1f8aabd5472fcdffaabde54369a9774ed57ab6faf3aca9"} Jul 08 19:12:16.276088 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:16.276009 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:16.276233 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:16.276167 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:12:16.276268 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:16.276245 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls podName:9e82e9f4-ccde-4b94-91f9-c5ed181ae83c nodeName:}" failed. No retries permitted until 2026-07-08 19:12:17.276230077 +0000 UTC m=+45.945481746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-w9db9" (UID: "9e82e9f4-ccde-4b94-91f9-c5ed181ae83c") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:12:16.376559 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:16.376523 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:16.376719 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:16.376629 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:12:16.376719 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:16.376641 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f8fc8f87d-p5vqg: secret "image-registry-tls" not found Jul 08 19:12:16.376719 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:16.376691 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls podName:f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7 nodeName:}" failed. No retries permitted until 2026-07-08 19:12:17.376676221 +0000 UTC m=+46.045927889 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls") pod "image-registry-f8fc8f87d-p5vqg" (UID: "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7") : secret "image-registry-tls" not found Jul 08 19:12:17.286252 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:17.285588 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:17.286252 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:17.285777 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:12:17.286252 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:17.285843 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls podName:9e82e9f4-ccde-4b94-91f9-c5ed181ae83c nodeName:}" failed. No retries permitted until 2026-07-08 19:12:19.285824534 +0000 UTC m=+47.955076204 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-w9db9" (UID: "9e82e9f4-ccde-4b94-91f9-c5ed181ae83c") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:12:17.387197 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:17.386566 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:17.387197 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:17.386730 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:12:17.387197 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:17.386746 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f8fc8f87d-p5vqg: secret "image-registry-tls" not found Jul 08 19:12:17.387197 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:17.386811 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls podName:f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7 nodeName:}" failed. No retries permitted until 2026-07-08 19:12:19.386790728 +0000 UTC m=+48.056042435 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls") pod "image-registry-f8fc8f87d-p5vqg" (UID: "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7") : secret "image-registry-tls" not found Jul 08 19:12:18.624157 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:18.624116 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:19.303281 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:19.303244 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:19.303478 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:19.303437 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:12:19.303549 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:19.303508 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls podName:9e82e9f4-ccde-4b94-91f9-c5ed181ae83c nodeName:}" failed. No retries permitted until 2026-07-08 19:12:23.303487665 +0000 UTC m=+51.972739370 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-w9db9" (UID: "9e82e9f4-ccde-4b94-91f9-c5ed181ae83c") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:12:19.404681 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:19.404638 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:19.404865 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:19.404826 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:12:19.404865 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:19.404851 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f8fc8f87d-p5vqg: secret "image-registry-tls" not found Jul 08 19:12:19.404995 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:19.404912 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls podName:f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7 nodeName:}" failed. No retries permitted until 2026-07-08 19:12:23.404892437 +0000 UTC m=+52.074144109 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls") pod "image-registry-f8fc8f87d-p5vqg" (UID: "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7") : secret "image-registry-tls" not found Jul 08 19:12:22.250323 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.250288 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" event={"ID":"1538d1d4-a106-4aa3-9caa-153393cfe60c","Type":"ContainerStarted","Data":"28b3ed485771f4075d99d207b1c4219ced1004410a93d548f5f9b858582bfd74"} Jul 08 19:12:22.251936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.251916 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/0.log" Jul 08 19:12:22.252045 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.251950 2563 generic.go:356] "Generic (PLEG): container finished" podID="e1b57420-0780-41af-8d72-56425fe6a28d" containerID="fbefd6e95d79529b471112039c11cc0e3c67eef58d58a3d4c96b4a21ffa662a7" exitCode=255 Jul 08 19:12:22.252087 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.252039 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tppst" event={"ID":"e1b57420-0780-41af-8d72-56425fe6a28d","Type":"ContainerDied","Data":"fbefd6e95d79529b471112039c11cc0e3c67eef58d58a3d4c96b4a21ffa662a7"} Jul 08 19:12:22.252361 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.252339 2563 scope.go:117] "RemoveContainer" containerID="fbefd6e95d79529b471112039c11cc0e3c67eef58d58a3d4c96b4a21ffa662a7" Jul 08 19:12:22.255922 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.255860 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" event={"ID":"b42ce879-9228-489a-a7a8-d12970b3cd3a","Type":"ContainerStarted","Data":"bc421569955eb275fcf53495bcb3ee0197fd62012e5005d20e31a0dcad971685"} Jul 08 19:12:22.257456 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.257433 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" event={"ID":"c0e7870b-512b-4301-8ebc-3479c3ac23d3","Type":"ContainerStarted","Data":"419300872e910475ed4764f6f95a5e043c0b83b6265a717401d85bb4e08bbf4f"} Jul 08 19:12:22.294740 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.294653 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" podStartSLOduration=1.396760811 podStartE2EDuration="7.294587418s" totalImagesPullingTime="5.897826607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:22.293785113 +0000 UTC m=+50.963036804" watchObservedRunningTime="2026-07-08 19:12:22.294587418 +0000 UTC m=+50.963839109" Jul 08 19:12:22.331412 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.330959 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" podStartSLOduration=1.4626349730000001 podStartE2EDuration="7.330939967s" totalImagesPullingTime="5.868304994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:22.311869876 +0000 UTC m=+50.981121567" watchObservedRunningTime="2026-07-08 19:12:22.330939967 +0000 UTC m=+51.000191861" Jul 08 19:12:22.532856 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.532821 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:22.533049 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:22.533010 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:12:22.533104 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:22.533088 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls podName:51e8638f-de06-407c-9625-327c48e1157d nodeName:}" failed. No retries permitted until 2026-07-08 19:12:38.533066788 +0000 UTC m=+67.202318461 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls") pod "dns-default-7bqln" (UID: "51e8638f-de06-407c-9625-327c48e1157d") : secret "dns-default-metrics-tls" not found Jul 08 19:12:22.952580 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.952485 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" podStartSLOduration=2.157768635 podStartE2EDuration="7.95246306s" totalImagesPullingTime="5.794694425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:22.33045491 +0000 UTC m=+50.999706600" watchObservedRunningTime="2026-07-08 19:12:22.95246306 +0000 UTC m=+51.621714750" Jul 08 19:12:22.953288 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.953264 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lptmt"] Jul 08 19:12:22.961689 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.961665 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:22.973626 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.973601 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 19:12:22.973720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.973649 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 19:12:22.973914 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:22.973899 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qc64c\"" Jul 08 19:12:23.036727 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.036688 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.036876 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.036742 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/745295a2-6e6e-4969-bbbf-ea58019b984a-crio-socket\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.036876 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.036815 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6qkq\" (UniqueName: \"kubernetes.io/projected/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-api-access-j6qkq\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.037010 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.036982 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.037050 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.037023 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/745295a2-6e6e-4969-bbbf-ea58019b984a-data-volume\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.058218 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.058183 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:23.097737 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.097704 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lptmt"] Jul 08 19:12:23.137886 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.137850 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.137886 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.137886 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/745295a2-6e6e-4969-bbbf-ea58019b984a-data-volume\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.138111 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.137907 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.138111 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.137935 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/745295a2-6e6e-4969-bbbf-ea58019b984a-crio-socket\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.138111 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.137995 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qkq\" (UniqueName: \"kubernetes.io/projected/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-api-access-j6qkq\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.138111 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:23.138013 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:12:23.138111 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:23.138016 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:12:23.138111 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:23.138075 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls podName:745295a2-6e6e-4969-bbbf-ea58019b984a nodeName:}" failed. No retries permitted until 2026-07-08 19:12:23.638057747 +0000 UTC m=+52.307309445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lptmt" (UID: "745295a2-6e6e-4969-bbbf-ea58019b984a") : secret "insights-runtime-extractor-tls" not found Jul 08 19:12:23.138357 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:23.138166 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm podName:745295a2-6e6e-4969-bbbf-ea58019b984a nodeName:}" failed. No retries permitted until 2026-07-08 19:12:23.638145199 +0000 UTC m=+52.307396881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lptmt" (UID: "745295a2-6e6e-4969-bbbf-ea58019b984a") : configmap "kube-rbac-proxy" not found Jul 08 19:12:23.138357 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.138207 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/745295a2-6e6e-4969-bbbf-ea58019b984a-crio-socket\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.138357 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.138253 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/745295a2-6e6e-4969-bbbf-ea58019b984a-data-volume\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.146405 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.146363 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:23.147736 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.147706 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6qkq\" (UniqueName: \"kubernetes.io/projected/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-api-access-j6qkq\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.242816 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.242745 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:23.262631 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.262605 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-btcbl" event={"ID":"83db0d91-71be-4616-bbfb-02c836423f7b","Type":"ContainerStarted","Data":"a3b65f4a59a4648f0b19e5af5b98019e5d5c3e1a2e9c88b33560ae6be97f0edc"} Jul 08 19:12:23.264527 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.264507 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:12:23.264895 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.264879 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/0.log" Jul 08 19:12:23.265004 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.264911 2563 generic.go:356] "Generic (PLEG): container finished" podID="e1b57420-0780-41af-8d72-56425fe6a28d" containerID="19ea2bc0576c978ff0aa0919a19b2cb45409c86f4da3e0f35f87b0f773a37ce6" exitCode=255 Jul 08 19:12:23.265072 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.265011 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tppst" event={"ID":"e1b57420-0780-41af-8d72-56425fe6a28d","Type":"ContainerDied","Data":"19ea2bc0576c978ff0aa0919a19b2cb45409c86f4da3e0f35f87b0f773a37ce6"} Jul 08 19:12:23.265118 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.265069 2563 scope.go:117] "RemoveContainer" containerID="fbefd6e95d79529b471112039c11cc0e3c67eef58d58a3d4c96b4a21ffa662a7" Jul 08 19:12:23.265401 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.265308 2563 scope.go:117] "RemoveContainer" containerID="19ea2bc0576c978ff0aa0919a19b2cb45409c86f4da3e0f35f87b0f773a37ce6" Jul 08 19:12:23.265552 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:23.265508 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-tppst_openshift-console-operator(e1b57420-0780-41af-8d72-56425fe6a28d)\"" pod="openshift-console-operator/console-operator-68b757fb67-tppst" podUID="e1b57420-0780-41af-8d72-56425fe6a28d" Jul 08 19:12:23.282339 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.282295 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-btcbl" podStartSLOduration=2.266204709 podStartE2EDuration="8.282277759s" totalImagesPullingTime="6.01607305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:23.281023969 +0000 UTC m=+51.950275683" watchObservedRunningTime="2026-07-08 19:12:23.282277759 +0000 UTC m=+51.951529451" Jul 08 19:12:23.307740 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.307705 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:23.340128 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.340102 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:23.340242 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:23.340211 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:12:23.340303 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:23.340271 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls podName:9e82e9f4-ccde-4b94-91f9-c5ed181ae83c nodeName:}" failed. No retries permitted until 2026-07-08 19:12:31.340253981 +0000 UTC m=+60.009505648 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-w9db9" (UID: "9e82e9f4-ccde-4b94-91f9-c5ed181ae83c") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:12:23.441469 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.441440 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:23.441619 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:23.441606 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:12:23.441661 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:23.441622 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f8fc8f87d-p5vqg: secret "image-registry-tls" not found Jul 08 19:12:23.441694 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:23.441681 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls podName:f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7 nodeName:}" failed. No retries permitted until 2026-07-08 19:12:31.441665551 +0000 UTC m=+60.110917240 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls") pod "image-registry-f8fc8f87d-p5vqg" (UID: "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7") : secret "image-registry-tls" not found Jul 08 19:12:23.644289 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.644250 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.644289 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.644290 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:23.644563 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:23.644422 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:12:23.644563 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:23.644442 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:12:23.644563 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:23.644474 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm podName:745295a2-6e6e-4969-bbbf-ea58019b984a nodeName:}" failed. No retries permitted until 2026-07-08 19:12:24.644460593 +0000 UTC m=+53.313712261 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lptmt" (UID: "745295a2-6e6e-4969-bbbf-ea58019b984a") : configmap "kube-rbac-proxy" not found Jul 08 19:12:23.644563 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:23.644517 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls podName:745295a2-6e6e-4969-bbbf-ea58019b984a nodeName:}" failed. No retries permitted until 2026-07-08 19:12:24.644499776 +0000 UTC m=+53.313751443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lptmt" (UID: "745295a2-6e6e-4969-bbbf-ea58019b984a") : secret "insights-runtime-extractor-tls" not found Jul 08 19:12:23.934287 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:23.934179 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:24.037136 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:24.037103 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:24.270294 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:24.270271 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:12:24.270751 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:24.270731 2563 scope.go:117] "RemoveContainer" containerID="19ea2bc0576c978ff0aa0919a19b2cb45409c86f4da3e0f35f87b0f773a37ce6" Jul 08 19:12:24.270887 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:24.270871 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-tppst_openshift-console-operator(e1b57420-0780-41af-8d72-56425fe6a28d)\"" pod="openshift-console-operator/console-operator-68b757fb67-tppst" podUID="e1b57420-0780-41af-8d72-56425fe6a28d" Jul 08 19:12:24.524832 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:24.524765 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ppkjh_237823e9-8de2-4851-b96f-ec6cf742d306/dns-node-resolver/0.log" Jul 08 19:12:24.591122 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:24.591086 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:24.647120 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:24.647058 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:24.652191 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:24.652170 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:24.652293 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:24.652198 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:24.652293 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:24.652281 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:12:24.652387 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:24.652307 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:12:24.652387 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:24.652353 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm podName:745295a2-6e6e-4969-bbbf-ea58019b984a nodeName:}" failed. No retries permitted until 2026-07-08 19:12:26.65233621 +0000 UTC m=+55.321587880 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lptmt" (UID: "745295a2-6e6e-4969-bbbf-ea58019b984a") : configmap "kube-rbac-proxy" not found Jul 08 19:12:24.652480 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:24.652433 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls podName:745295a2-6e6e-4969-bbbf-ea58019b984a nodeName:}" failed. No retries permitted until 2026-07-08 19:12:26.652419893 +0000 UTC m=+55.321671561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lptmt" (UID: "745295a2-6e6e-4969-bbbf-ea58019b984a") : secret "insights-runtime-extractor-tls" not found Jul 08 19:12:24.747026 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:24.746990 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:25.541158 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.541125 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-fps7c"] Jul 08 19:12:25.573309 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.573283 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-fps7c"] Jul 08 19:12:25.573495 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.573369 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-fps7c" Jul 08 19:12:25.576399 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.576363 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 08 19:12:25.576533 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.576417 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 08 19:12:25.577462 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.577447 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-2pfz9\"" Jul 08 19:12:25.577547 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.577484 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 08 19:12:25.577586 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.577559 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 08 19:12:25.587715 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.587693 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:25.661968 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.661940 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8174867e-46db-4a02-a4b9-651099a4c447-signing-key\") pod \"service-ca-758f599c5-fps7c\" (UID: \"8174867e-46db-4a02-a4b9-651099a4c447\") " pod="openshift-service-ca/service-ca-758f599c5-fps7c" Jul 08 19:12:25.662089 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.662067 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8174867e-46db-4a02-a4b9-651099a4c447-signing-cabundle\") pod \"service-ca-758f599c5-fps7c\" (UID: \"8174867e-46db-4a02-a4b9-651099a4c447\") " pod="openshift-service-ca/service-ca-758f599c5-fps7c" Jul 08 19:12:25.662131 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.662097 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgrdd\" (UniqueName: \"kubernetes.io/projected/8174867e-46db-4a02-a4b9-651099a4c447-kube-api-access-vgrdd\") pod \"service-ca-758f599c5-fps7c\" (UID: \"8174867e-46db-4a02-a4b9-651099a4c447\") " pod="openshift-service-ca/service-ca-758f599c5-fps7c" Jul 08 19:12:25.685770 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.685740 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:25.725650 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.725625 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n2kkx_352fc82d-a3e3-4157-ae41-5eade870e4f0/node-ca/0.log" Jul 08 19:12:25.763110 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.763072 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8174867e-46db-4a02-a4b9-651099a4c447-signing-key\") pod \"service-ca-758f599c5-fps7c\" (UID: \"8174867e-46db-4a02-a4b9-651099a4c447\") " pod="openshift-service-ca/service-ca-758f599c5-fps7c" Jul 08 19:12:25.763317 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.763301 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8174867e-46db-4a02-a4b9-651099a4c447-signing-cabundle\") pod \"service-ca-758f599c5-fps7c\" (UID: \"8174867e-46db-4a02-a4b9-651099a4c447\") " pod="openshift-service-ca/service-ca-758f599c5-fps7c" Jul 08 19:12:25.763418 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.763351 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgrdd\" (UniqueName: \"kubernetes.io/projected/8174867e-46db-4a02-a4b9-651099a4c447-kube-api-access-vgrdd\") pod \"service-ca-758f599c5-fps7c\" (UID: \"8174867e-46db-4a02-a4b9-651099a4c447\") " pod="openshift-service-ca/service-ca-758f599c5-fps7c" Jul 08 19:12:25.763903 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.763883 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8174867e-46db-4a02-a4b9-651099a4c447-signing-cabundle\") pod \"service-ca-758f599c5-fps7c\" (UID: \"8174867e-46db-4a02-a4b9-651099a4c447\") " pod="openshift-service-ca/service-ca-758f599c5-fps7c" Jul 08 19:12:25.765526 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.765508 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8174867e-46db-4a02-a4b9-651099a4c447-signing-key\") pod \"service-ca-758f599c5-fps7c\" (UID: \"8174867e-46db-4a02-a4b9-651099a4c447\") " pod="openshift-service-ca/service-ca-758f599c5-fps7c" Jul 08 19:12:25.771548 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.771519 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgrdd\" (UniqueName: \"kubernetes.io/projected/8174867e-46db-4a02-a4b9-651099a4c447-kube-api-access-vgrdd\") pod \"service-ca-758f599c5-fps7c\" (UID: \"8174867e-46db-4a02-a4b9-651099a4c447\") " pod="openshift-service-ca/service-ca-758f599c5-fps7c" Jul 08 19:12:25.785664 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.785639 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:25.882568 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.882479 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-fps7c" Jul 08 19:12:25.945490 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.945460 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:25.945624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.945533 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:25.946014 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:25.945995 2563 scope.go:117] "RemoveContainer" containerID="19ea2bc0576c978ff0aa0919a19b2cb45409c86f4da3e0f35f87b0f773a37ce6" Jul 08 19:12:25.946238 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:25.946209 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-tppst_openshift-console-operator(e1b57420-0780-41af-8d72-56425fe6a28d)\"" pod="openshift-console-operator/console-operator-68b757fb67-tppst" podUID="e1b57420-0780-41af-8d72-56425fe6a28d" Jul 08 19:12:26.018511 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:26.018475 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-fps7c"] Jul 08 19:12:26.021100 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:26.021070 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8174867e_46db_4a02_a4b9_651099a4c447.slice/crio-365e544b14f0c961322b5e83614e13fc485bfc73f6fa818b37b35611d3eb0840 WatchSource:0}: Error finding container 365e544b14f0c961322b5e83614e13fc485bfc73f6fa818b37b35611d3eb0840: Status 404 returned error can't find the container with id 365e544b14f0c961322b5e83614e13fc485bfc73f6fa818b37b35611d3eb0840 Jul 08 19:12:26.124785 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:26.124755 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hpgnx_a8262b98-73be-424f-9ac6-5f65172997af/serve-healthcheck-canary/0.log" Jul 08 19:12:26.282793 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:26.282757 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-fps7c" event={"ID":"8174867e-46db-4a02-a4b9-651099a4c447","Type":"ContainerStarted","Data":"970ebc9f7aa0cf3951b302cb3604f0b6d3e1b4a5f270b2b519f9ad37ea0caea3"} Jul 08 19:12:26.282942 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:26.282830 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-fps7c" event={"ID":"8174867e-46db-4a02-a4b9-651099a4c447","Type":"ContainerStarted","Data":"365e544b14f0c961322b5e83614e13fc485bfc73f6fa818b37b35611d3eb0840"} Jul 08 19:12:26.308236 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:26.308188 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-fps7c" podStartSLOduration=1.308172916 podStartE2EDuration="1.308172916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:26.306054732 +0000 UTC m=+54.975306418" watchObservedRunningTime="2026-07-08 19:12:26.308172916 +0000 UTC m=+54.977424606" Jul 08 19:12:26.671631 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:26.671535 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:26.671631 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:26.671589 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:26.672055 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:26.671689 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:12:26.672055 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:26.671697 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:12:26.672055 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:26.671755 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm podName:745295a2-6e6e-4969-bbbf-ea58019b984a nodeName:}" failed. No retries permitted until 2026-07-08 19:12:30.671737092 +0000 UTC m=+59.340988765 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lptmt" (UID: "745295a2-6e6e-4969-bbbf-ea58019b984a") : configmap "kube-rbac-proxy" not found Jul 08 19:12:26.672055 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:26.671773 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls podName:745295a2-6e6e-4969-bbbf-ea58019b984a nodeName:}" failed. No retries permitted until 2026-07-08 19:12:30.671763689 +0000 UTC m=+59.341015362 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lptmt" (UID: "745295a2-6e6e-4969-bbbf-ea58019b984a") : secret "insights-runtime-extractor-tls" not found Jul 08 19:12:26.912854 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:26.912819 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:27.012820 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:27.012783 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:27.148152 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:27.148115 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:27.264741 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:27.264618 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:27.366793 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:27.366762 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:27.444706 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:27.444669 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:27.605754 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:27.605662 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:28.572054 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:28.572023 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:28.976174 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:28.976084 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:29.170758 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:29.170722 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:29.373299 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:29.373258 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:29.779847 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:29.779815 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:30.376098 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:30.376048 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:30.557863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:30.557833 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:30.630921 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:30.630836 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:30.709522 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:30.709495 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:30.709701 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:30.709528 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:30.709701 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:30.709649 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:12:30.709701 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:30.709661 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:12:30.709804 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:30.709713 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm podName:745295a2-6e6e-4969-bbbf-ea58019b984a nodeName:}" failed. No retries permitted until 2026-07-08 19:12:38.709699013 +0000 UTC m=+67.378950680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lptmt" (UID: "745295a2-6e6e-4969-bbbf-ea58019b984a") : configmap "kube-rbac-proxy" not found Jul 08 19:12:30.709804 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:30.709727 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls podName:745295a2-6e6e-4969-bbbf-ea58019b984a nodeName:}" failed. No retries permitted until 2026-07-08 19:12:38.709721492 +0000 UTC m=+67.378973160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lptmt" (UID: "745295a2-6e6e-4969-bbbf-ea58019b984a") : secret "insights-runtime-extractor-tls" not found Jul 08 19:12:30.774564 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:30.774530 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:30.875570 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:30.875543 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:31.174090 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:31.174056 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:31.372300 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:31.372272 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:31.415589 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:31.415557 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:31.415764 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:31.415737 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:12:31.415822 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:12:31.415811 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls podName:9e82e9f4-ccde-4b94-91f9-c5ed181ae83c nodeName:}" failed. No retries permitted until 2026-07-08 19:12:47.415790535 +0000 UTC m=+76.085042209 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-w9db9" (UID: "9e82e9f4-ccde-4b94-91f9-c5ed181ae83c") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:12:31.473537 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:31.473451 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:31.516405 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:31.516361 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:31.518699 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:31.518665 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls\") pod \"image-registry-f8fc8f87d-p5vqg\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:31.669805 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:31.669768 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:31.771840 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:31.771802 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:31.810189 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:31.810159 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f8fc8f87d-p5vqg"] Jul 08 19:12:31.813248 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:31.813223 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7b1e1c1_67b3_4846_a7d8_e4e7b2f4d3f7.slice/crio-c79be78f103c17cb3ea52a6a669f41d26954358194415ee711697d725d1a86d6 WatchSource:0}: Error finding container c79be78f103c17cb3ea52a6a669f41d26954358194415ee711697d725d1a86d6: Status 404 returned error can't find the container with id c79be78f103c17cb3ea52a6a669f41d26954358194415ee711697d725d1a86d6 Jul 08 19:12:32.254625 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:32.254590 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:32.306146 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:32.306111 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" event={"ID":"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7","Type":"ContainerStarted","Data":"dc3dcf31878be0b7697db057fb6e699f423175ed66ece45290fd9594f4de931e"} Jul 08 19:12:32.306325 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:32.306169 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" event={"ID":"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7","Type":"ContainerStarted","Data":"c79be78f103c17cb3ea52a6a669f41d26954358194415ee711697d725d1a86d6"} Jul 08 19:12:32.306325 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:32.306209 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:32.327206 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:32.327164 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" podStartSLOduration=17.327152041 podStartE2EDuration="17.327152041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:32.326805392 +0000 UTC m=+60.996057111" watchObservedRunningTime="2026-07-08 19:12:32.327152041 +0000 UTC m=+60.996403730" Jul 08 19:12:32.331231 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:32.331206 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:32.385946 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:32.385893 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:32.815670 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:32.815632 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:32.868076 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:32.868045 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:33.172178 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:33.172079 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:33.195237 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:33.195214 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcr4l" Jul 08 19:12:33.380890 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:33.380858 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:33.971048 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:33.971015 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:34.373476 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:34.373436 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:34.772292 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:34.772254 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:35.173810 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:35.173718 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:35.371901 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:35.371868 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:35.976305 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:35.976275 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:36.137713 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:36.137681 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:36.374201 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:36.374163 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:36.777934 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:36.777900 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:36.978136 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:36.978096 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:37.379331 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.379296 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:37.566982 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.566950 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:12:37.569869 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.569851 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 19:12:37.580182 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.580150 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:37.580507 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.580492 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/508e47f4-f845-4e04-88dd-1ab13d22d3b8-original-pull-secret\") pod \"global-pull-secret-syncer-z22h9\" (UID: \"508e47f4-f845-4e04-88dd-1ab13d22d3b8\") " pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:12:37.667850 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.667744 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs\") pod \"network-metrics-daemon-xthzl\" (UID: \"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2\") " pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:12:37.667850 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.667808 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l7zv\" (UniqueName: \"kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv\") pod \"network-check-target-xg866\" (UID: \"9d287a39-6145-4cc3-b16f-226c4d738dd9\") " pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:12:37.670274 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.670255 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l7zv\" (UniqueName: \"kubernetes.io/projected/9d287a39-6145-4cc3-b16f-226c4d738dd9-kube-api-access-9l7zv\") pod \"network-check-target-xg866\" (UID: \"9d287a39-6145-4cc3-b16f-226c4d738dd9\") " pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:12:37.670935 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.670913 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 19:12:37.680002 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.679981 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa141408-9f29-44b9-9f55-dcb1fdd5bdf2-metrics-certs\") pod \"network-metrics-daemon-xthzl\" (UID: \"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2\") " pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:12:37.823782 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.823750 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8qd8q\"" Jul 08 19:12:37.826913 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.826897 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z22h9" Jul 08 19:12:37.831463 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.831444 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xthzl" Jul 08 19:12:37.899093 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.898202 2563 scope.go:117] "RemoveContainer" containerID="19ea2bc0576c978ff0aa0919a19b2cb45409c86f4da3e0f35f87b0f773a37ce6" Jul 08 19:12:37.911632 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.911615 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-92l7d\"" Jul 08 19:12:37.919244 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.919179 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:12:37.971443 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.971170 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z22h9"] Jul 08 19:12:37.978164 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:37.978125 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod508e47f4_f845_4e04_88dd_1ab13d22d3b8.slice/crio-befd6505f632d796afa553d93a1c77909ade173c84e380b6543c40257d6b5209 WatchSource:0}: Error finding container befd6505f632d796afa553d93a1c77909ade173c84e380b6543c40257d6b5209: Status 404 returned error can't find the container with id befd6505f632d796afa553d93a1c77909ade173c84e380b6543c40257d6b5209 Jul 08 19:12:37.989529 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:37.989294 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xthzl"] Jul 08 19:12:37.992026 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:37.991997 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa141408_9f29_44b9_9f55_dcb1fdd5bdf2.slice/crio-7eb9731692b0c6e372e15310a2b495629b4515c75835a752b9b4b11001b9280c WatchSource:0}: Error finding container 7eb9731692b0c6e372e15310a2b495629b4515c75835a752b9b4b11001b9280c: Status 404 returned error can't find the container with id 7eb9731692b0c6e372e15310a2b495629b4515c75835a752b9b4b11001b9280c Jul 08 19:12:38.051750 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.051716 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xg866"] Jul 08 19:12:38.054838 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:38.054814 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d287a39_6145_4cc3_b16f_226c4d738dd9.slice/crio-1682b2a7c8b705956e325a32829551591680b47021791e7a84cef462a25f3823 WatchSource:0}: Error finding container 1682b2a7c8b705956e325a32829551591680b47021791e7a84cef462a25f3823: Status 404 returned error can't find the container with id 1682b2a7c8b705956e325a32829551591680b47021791e7a84cef462a25f3823 Jul 08 19:12:38.319942 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.319905 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:38.326716 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.326686 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xg866" event={"ID":"9d287a39-6145-4cc3-b16f-226c4d738dd9","Type":"ContainerStarted","Data":"8e1a68b517a9e01851b9082d444adcfd9af1d8d3b1af1cae083d3413936ff73b"} Jul 08 19:12:38.326865 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.326739 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xg866" event={"ID":"9d287a39-6145-4cc3-b16f-226c4d738dd9","Type":"ContainerStarted","Data":"1682b2a7c8b705956e325a32829551591680b47021791e7a84cef462a25f3823"} Jul 08 19:12:38.326865 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.326800 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:12:38.328101 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.328074 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xthzl" event={"ID":"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2","Type":"ContainerStarted","Data":"7eb9731692b0c6e372e15310a2b495629b4515c75835a752b9b4b11001b9280c"} Jul 08 19:12:38.329212 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.329188 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z22h9" event={"ID":"508e47f4-f845-4e04-88dd-1ab13d22d3b8","Type":"ContainerStarted","Data":"befd6505f632d796afa553d93a1c77909ade173c84e380b6543c40257d6b5209"} Jul 08 19:12:38.331015 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.330998 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:12:38.331081 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.331032 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tppst" event={"ID":"e1b57420-0780-41af-8d72-56425fe6a28d","Type":"ContainerStarted","Data":"82fe92e2ddb80e3e779889b96d67d861e9c2ef4d78b8798fe51f26306c403320"} Jul 08 19:12:38.331273 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.331248 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:38.345367 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.345329 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xg866" podStartSLOduration=67.345319117 podStartE2EDuration="1m7.345319117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:38.344825672 +0000 UTC m=+67.014077362" watchObservedRunningTime="2026-07-08 19:12:38.345319117 +0000 UTC m=+67.014570808" Jul 08 19:12:38.362909 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.362873 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-tppst" podStartSLOduration=17.55504229 podStartE2EDuration="23.362862526s" totalImagesPullingTime="5.807820236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:38.362230933 +0000 UTC m=+67.031482622" watchObservedRunningTime="2026-07-08 19:12:38.362862526 +0000 UTC m=+67.032114226" Jul 08 19:12:38.378785 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.378762 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-tppst" Jul 08 19:12:38.557573 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.557535 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:38.579133 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.578642 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:38.581793 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.581760 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51e8638f-de06-407c-9625-327c48e1157d-metrics-tls\") pod \"dns-default-7bqln\" (UID: \"51e8638f-de06-407c-9625-327c48e1157d\") " pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:38.654305 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.654244 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:38.779752 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.779711 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:38.779931 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.779763 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:38.780414 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.780363 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/745295a2-6e6e-4969-bbbf-ea58019b984a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:38.782623 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.782598 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/745295a2-6e6e-4969-bbbf-ea58019b984a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lptmt\" (UID: \"745295a2-6e6e-4969-bbbf-ea58019b984a\") " pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:38.819334 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.819302 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4xh72\"" Jul 08 19:12:38.827676 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.827648 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:38.830203 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.830170 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:38.875153 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.875114 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qc64c\"" Jul 08 19:12:38.882835 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.882812 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lptmt" Jul 08 19:12:38.889187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.889014 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:38.987622 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:38.987591 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7bqln"] Jul 08 19:12:38.992532 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:38.992477 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e8638f_de06_407c_9625_327c48e1157d.slice/crio-c9d13d06e93540502198eb5604e1d05eb3ce86f44728bd5d527232c1618d2b23 WatchSource:0}: Error finding container c9d13d06e93540502198eb5604e1d05eb3ce86f44728bd5d527232c1618d2b23: Status 404 returned error can't find the container with id c9d13d06e93540502198eb5604e1d05eb3ce86f44728bd5d527232c1618d2b23 Jul 08 19:12:39.040192 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:39.040158 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lptmt"] Jul 08 19:12:39.044039 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:39.044008 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod745295a2_6e6e_4969_bbbf_ea58019b984a.slice/crio-9cbfe3bd5fe9584b3505ec4c5bfff45da734d81bb8ff6aa3ba6080212af2b003 WatchSource:0}: Error finding container 9cbfe3bd5fe9584b3505ec4c5bfff45da734d81bb8ff6aa3ba6080212af2b003: Status 404 returned error can't find the container with id 9cbfe3bd5fe9584b3505ec4c5bfff45da734d81bb8ff6aa3ba6080212af2b003 Jul 08 19:12:39.057172 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:39.057141 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:39.239268 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:39.239186 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:39.298650 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:39.298614 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:39.337656 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:39.337623 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7bqln" event={"ID":"51e8638f-de06-407c-9625-327c48e1157d","Type":"ContainerStarted","Data":"c9d13d06e93540502198eb5604e1d05eb3ce86f44728bd5d527232c1618d2b23"} Jul 08 19:12:39.339408 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:39.338909 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lptmt" event={"ID":"745295a2-6e6e-4969-bbbf-ea58019b984a","Type":"ContainerStarted","Data":"9cbfe3bd5fe9584b3505ec4c5bfff45da734d81bb8ff6aa3ba6080212af2b003"} Jul 08 19:12:39.441918 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:39.441875 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:39.869232 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:39.869179 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:39.972326 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:39.972289 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:40.073171 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:40.073110 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:40.144726 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:40.144630 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:40.346793 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:40.346693 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xthzl" event={"ID":"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2","Type":"ContainerStarted","Data":"89207cafea1f19989c584e26a850650463cf93e84745e7ad0b2193826407d017"} Jul 08 19:12:40.346793 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:40.346748 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xthzl" event={"ID":"fa141408-9f29-44b9-9f55-dcb1fdd5bdf2","Type":"ContainerStarted","Data":"fc220c94f333ff55ab4633b3997f8d4cf33ca820e9e8794251be3a745021988a"} Jul 08 19:12:40.365969 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:40.365913 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xthzl" podStartSLOduration=67.042011893 podStartE2EDuration="1m8.365892011s" totalImagesPullingTime="1.323880118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:40.363244967 +0000 UTC m=+69.032496658" watchObservedRunningTime="2026-07-08 19:12:40.365892011 +0000 UTC m=+69.035143702" Jul 08 19:12:40.561057 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:40.561021 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:40.662968 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:40.662916 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:41.668056 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:41.668020 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:41.733787 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:41.733747 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:46.368914 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:46.368825 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z22h9" event={"ID":"508e47f4-f845-4e04-88dd-1ab13d22d3b8","Type":"ContainerStarted","Data":"cb9d9dbb9ebfdf0f9d05c1ade77151a945b847f4c3b662eef97ac722350569f1"} Jul 08 19:12:46.370707 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:46.370672 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7bqln" event={"ID":"51e8638f-de06-407c-9625-327c48e1157d","Type":"ContainerStarted","Data":"51176346068cb120c325355486ef722996ddfbef85555b9c1c19e40e23645d65"} Jul 08 19:12:46.370848 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:46.370729 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7bqln" event={"ID":"51e8638f-de06-407c-9625-327c48e1157d","Type":"ContainerStarted","Data":"6ca02c87d59faa8f7ba8d19459488f8069304f4b5f2a1b9a21992c6649ab1381"} Jul 08 19:12:46.370848 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:46.370760 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:46.372322 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:46.372299 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lptmt" event={"ID":"745295a2-6e6e-4969-bbbf-ea58019b984a","Type":"ContainerStarted","Data":"212deec1d5bb26d64cd1b61934c059c1f87f4436491555d120075f74ad06b6ce"} Jul 08 19:12:46.372434 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:46.372340 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lptmt" event={"ID":"745295a2-6e6e-4969-bbbf-ea58019b984a","Type":"ContainerStarted","Data":"f143329a6275d6fa125ec98ec52a94922b87e1aed4ee37f4c5c9f5d95959f43a"} Jul 08 19:12:46.386271 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:46.386229 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-z22h9" podStartSLOduration=66.964058686 podStartE2EDuration="1m14.386215857s" totalImagesPullingTime="7.422157171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:46.385269191 +0000 UTC m=+75.054520882" watchObservedRunningTime="2026-07-08 19:12:46.386215857 +0000 UTC m=+75.055467525" Jul 08 19:12:46.403250 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:46.403207 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7bqln" podStartSLOduration=34.019683074 podStartE2EDuration="40.403193207s" totalImagesPullingTime="6.383510133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:46.402403904 +0000 UTC m=+75.071655594" watchObservedRunningTime="2026-07-08 19:12:46.403193207 +0000 UTC m=+75.072444920" Jul 08 19:12:47.454923 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:47.454882 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:47.457604 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:47.457578 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e82e9f4-ccde-4b94-91f9-c5ed181ae83c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-w9db9\" (UID: \"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:47.693321 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:47.693286 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-8vhgk\"" Jul 08 19:12:47.700559 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:47.700535 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" Jul 08 19:12:48.622182 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.622150 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-7ff64"] Jul 08 19:12:48.624996 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.624975 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b"] Jul 08 19:12:48.625817 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.625319 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-7ff64" Jul 08 19:12:48.627981 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.627895 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.629331 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.629206 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 08 19:12:48.629331 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.629254 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 08 19:12:48.629507 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.629490 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ld2jc\"" Jul 08 19:12:48.635226 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.635024 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 08 19:12:48.639078 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.639057 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 08 19:12:48.639356 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.639340 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 08 19:12:48.639824 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.639665 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 08 19:12:48.639824 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.639775 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 08 19:12:48.639921 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.639822 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 08 19:12:48.640188 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.640013 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 08 19:12:48.649218 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.649197 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b"] Jul 08 19:12:48.652572 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.651413 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-7ff64"] Jul 08 19:12:48.653077 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.653056 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f8fc8f87d-p5vqg"] Jul 08 19:12:48.659230 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.659125 2563 patch_prober.go:28] interesting pod/image-registry-f8fc8f87d-p5vqg container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 19:12:48.659230 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.659166 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" podUID="f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 19:12:48.712963 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.712936 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64b964b7cc-vpq7c"] Jul 08 19:12:48.719872 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.719854 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9"] Jul 08 19:12:48.719949 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.719940 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.721710 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:48.721687 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e82e9f4_ccde_4b94_91f9_c5ed181ae83c.slice/crio-80a559e622c57065e659b99d6e4ba72a3ce791d3fbe42710a219acc27dff3f0a WatchSource:0}: Error finding container 80a559e622c57065e659b99d6e4ba72a3ce791d3fbe42710a219acc27dff3f0a: Status 404 returned error can't find the container with id 80a559e622c57065e659b99d6e4ba72a3ce791d3fbe42710a219acc27dff3f0a Jul 08 19:12:48.729729 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.729706 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64b964b7cc-vpq7c"] Jul 08 19:12:48.767009 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.766976 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-hub\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.767009 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.767009 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.767183 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.767030 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-ca\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.767183 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.767082 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/98dcb86d-1429-4f0d-8740-0f4fa76916d5-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7ff64\" (UID: \"98dcb86d-1429-4f0d-8740-0f4fa76916d5\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7ff64" Jul 08 19:12:48.767183 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.767129 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.767183 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.767157 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hbbk\" (UniqueName: \"kubernetes.io/projected/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-kube-api-access-4hbbk\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.767301 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.767189 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/98dcb86d-1429-4f0d-8740-0f4fa76916d5-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-7ff64\" (UID: \"98dcb86d-1429-4f0d-8740-0f4fa76916d5\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7ff64" Jul 08 19:12:48.767301 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.767206 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.800117 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.800077 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:48.868519 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.868442 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hbbk\" (UniqueName: \"kubernetes.io/projected/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-kube-api-access-4hbbk\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.868519 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.868487 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd0826c4-4f05-4da4-89ab-ccf456cc1074-installation-pull-secrets\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.868519 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.868510 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/98dcb86d-1429-4f0d-8740-0f4fa76916d5-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-7ff64\" (UID: \"98dcb86d-1429-4f0d-8740-0f4fa76916d5\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7ff64" Jul 08 19:12:48.868742 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.868526 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.868742 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.868582 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0826c4-4f05-4da4-89ab-ccf456cc1074-registry-tls\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.868742 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.868629 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg24r\" (UniqueName: \"kubernetes.io/projected/fd0826c4-4f05-4da4-89ab-ccf456cc1074-kube-api-access-jg24r\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.868742 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.868676 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd0826c4-4f05-4da4-89ab-ccf456cc1074-registry-certificates\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.868742 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.868707 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd0826c4-4f05-4da4-89ab-ccf456cc1074-image-registry-private-configuration\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.868917 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.868767 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-hub\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.868917 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.868797 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd0826c4-4f05-4da4-89ab-ccf456cc1074-trusted-ca\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.868917 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.868822 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.868917 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.868841 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd0826c4-4f05-4da4-89ab-ccf456cc1074-bound-sa-token\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.868917 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.868866 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-ca\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.869095 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.868981 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd0826c4-4f05-4da4-89ab-ccf456cc1074-ca-trust-extracted\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.869095 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.869076 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/98dcb86d-1429-4f0d-8740-0f4fa76916d5-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7ff64\" (UID: \"98dcb86d-1429-4f0d-8740-0f4fa76916d5\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7ff64" Jul 08 19:12:48.869169 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.869141 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.869391 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.869328 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/98dcb86d-1429-4f0d-8740-0f4fa76916d5-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-7ff64\" (UID: \"98dcb86d-1429-4f0d-8740-0f4fa76916d5\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7ff64" Jul 08 19:12:48.869933 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.869907 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.871273 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.871246 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.871395 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.871327 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-ca\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.871956 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.871934 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/98dcb86d-1429-4f0d-8740-0f4fa76916d5-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7ff64\" (UID: \"98dcb86d-1429-4f0d-8740-0f4fa76916d5\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7ff64" Jul 08 19:12:48.872059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.872027 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-hub\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.872100 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.872079 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.881821 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.881802 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hbbk\" (UniqueName: \"kubernetes.io/projected/20e87c68-5d08-4860-b03a-8d4cc3fdcbfb-kube-api-access-4hbbk\") pod \"cluster-proxy-proxy-agent-674c95dc5c-vd42b\" (UID: \"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.898918 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.898896 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:48.964286 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.964020 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-7ff64" Jul 08 19:12:48.970350 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.970321 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg24r\" (UniqueName: \"kubernetes.io/projected/fd0826c4-4f05-4da4-89ab-ccf456cc1074-kube-api-access-jg24r\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.970489 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.970354 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd0826c4-4f05-4da4-89ab-ccf456cc1074-registry-certificates\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.970489 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.970389 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd0826c4-4f05-4da4-89ab-ccf456cc1074-image-registry-private-configuration\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.970489 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.970410 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd0826c4-4f05-4da4-89ab-ccf456cc1074-trusted-ca\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.970489 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.970437 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd0826c4-4f05-4da4-89ab-ccf456cc1074-bound-sa-token\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.970489 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.970460 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd0826c4-4f05-4da4-89ab-ccf456cc1074-ca-trust-extracted\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.970760 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.970516 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd0826c4-4f05-4da4-89ab-ccf456cc1074-installation-pull-secrets\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.970760 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.970546 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0826c4-4f05-4da4-89ab-ccf456cc1074-registry-tls\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.970946 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.970919 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd0826c4-4f05-4da4-89ab-ccf456cc1074-ca-trust-extracted\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.971283 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.971256 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd0826c4-4f05-4da4-89ab-ccf456cc1074-registry-certificates\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.971392 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.971262 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd0826c4-4f05-4da4-89ab-ccf456cc1074-trusted-ca\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.972793 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.972767 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd0826c4-4f05-4da4-89ab-ccf456cc1074-image-registry-private-configuration\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.972872 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.972854 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0826c4-4f05-4da4-89ab-ccf456cc1074-registry-tls\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.972917 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.972871 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd0826c4-4f05-4da4-89ab-ccf456cc1074-installation-pull-secrets\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:48.978835 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.978818 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" Jul 08 19:12:48.991533 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:48.991512 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg24r\" (UniqueName: \"kubernetes.io/projected/fd0826c4-4f05-4da4-89ab-ccf456cc1074-kube-api-access-jg24r\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:49.006846 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.006820 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd0826c4-4f05-4da4-89ab-ccf456cc1074-bound-sa-token\") pod \"image-registry-64b964b7cc-vpq7c\" (UID: \"fd0826c4-4f05-4da4-89ab-ccf456cc1074\") " pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:49.019604 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.019573 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:49.038795 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.038761 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:49.117029 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.116982 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:49.141390 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.141337 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-7ff64"] Jul 08 19:12:49.143811 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:49.143787 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98dcb86d_1429_4f0d_8740_0f4fa76916d5.slice/crio-81b0c5c245549427aef4719bd33f0336f61d365af65c6f3562f6f70d43538e89 WatchSource:0}: Error finding container 81b0c5c245549427aef4719bd33f0336f61d365af65c6f3562f6f70d43538e89: Status 404 returned error can't find the container with id 81b0c5c245549427aef4719bd33f0336f61d365af65c6f3562f6f70d43538e89 Jul 08 19:12:49.160637 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.160617 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b"] Jul 08 19:12:49.163742 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:49.163713 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20e87c68_5d08_4860_b03a_8d4cc3fdcbfb.slice/crio-7e042f925b06f52e6f7cef31468fc9c68fe18ec935b26afb047c9122885bf0eb WatchSource:0}: Error finding container 7e042f925b06f52e6f7cef31468fc9c68fe18ec935b26afb047c9122885bf0eb: Status 404 returned error can't find the container with id 7e042f925b06f52e6f7cef31468fc9c68fe18ec935b26afb047c9122885bf0eb Jul 08 19:12:49.200308 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.200217 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64b964b7cc-vpq7c"] Jul 08 19:12:49.203026 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:12:49.202999 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0826c4_4f05_4da4_89ab_ccf456cc1074.slice/crio-8d0bc9153dc5b47c828c32d787ab230f63b6256b977a583668add3d6919534a3 WatchSource:0}: Error finding container 8d0bc9153dc5b47c828c32d787ab230f63b6256b977a583668add3d6919534a3: Status 404 returned error can't find the container with id 8d0bc9153dc5b47c828c32d787ab230f63b6256b977a583668add3d6919534a3 Jul 08 19:12:49.205816 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.205787 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:49.387354 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.387315 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-7ff64" event={"ID":"98dcb86d-1429-4f0d-8740-0f4fa76916d5","Type":"ContainerStarted","Data":"81b0c5c245549427aef4719bd33f0336f61d365af65c6f3562f6f70d43538e89"} Jul 08 19:12:49.388908 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.388881 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" event={"ID":"fd0826c4-4f05-4da4-89ab-ccf456cc1074","Type":"ContainerStarted","Data":"33e76f3ed80675e41ea1d5d64344c06f6b1ed16e2ff59cdfcf46944ffbc21e47"} Jul 08 19:12:49.389059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.388923 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" event={"ID":"fd0826c4-4f05-4da4-89ab-ccf456cc1074","Type":"ContainerStarted","Data":"8d0bc9153dc5b47c828c32d787ab230f63b6256b977a583668add3d6919534a3"} Jul 08 19:12:49.389059 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.388985 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:12:49.391077 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.391010 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lptmt" event={"ID":"745295a2-6e6e-4969-bbbf-ea58019b984a","Type":"ContainerStarted","Data":"2a102356678b045a446d202ff5aa70954fcb5b63f291c9a2cb92a78b7dc5e42b"} Jul 08 19:12:49.392226 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.392198 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" event={"ID":"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb","Type":"ContainerStarted","Data":"7e042f925b06f52e6f7cef31468fc9c68fe18ec935b26afb047c9122885bf0eb"} Jul 08 19:12:49.393342 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.393322 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" event={"ID":"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c","Type":"ContainerStarted","Data":"80a559e622c57065e659b99d6e4ba72a3ce791d3fbe42710a219acc27dff3f0a"} Jul 08 19:12:49.415256 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.414617 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" podStartSLOduration=1.414596008 podStartE2EDuration="1.414596008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:49.410725465 +0000 UTC m=+78.079977157" watchObservedRunningTime="2026-07-08 19:12:49.414596008 +0000 UTC m=+78.083847699" Jul 08 19:12:49.445088 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:49.445024 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lptmt" podStartSLOduration=18.016658913 podStartE2EDuration="27.444993519s" totalImagesPullingTime="9.428334606s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:12:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:49.443765342 +0000 UTC m=+78.113017033" watchObservedRunningTime="2026-07-08 19:12:49.444993519 +0000 UTC m=+78.114245210" Jul 08 19:12:50.672400 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:50.672290 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:50.742391 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:50.742344 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:51.681839 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:51.681791 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:51.746152 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:51.746115 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:54.160032 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.159986 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:54.263649 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.263611 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:54.328178 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.328141 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:54.416436 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.416332 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-7ff64" event={"ID":"98dcb86d-1429-4f0d-8740-0f4fa76916d5","Type":"ContainerStarted","Data":"63356f76d25e961987d07d28cf3cf2152aa4e71b23139f7df4257d6882643224"} Jul 08 19:12:54.418762 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.418728 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" event={"ID":"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb","Type":"ContainerStarted","Data":"cd390b444ae421c8d9cc283a5c5c02207e7866dd575f1ee23efcd43544f08cf1"} Jul 08 19:12:54.418916 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.418769 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" event={"ID":"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb","Type":"ContainerStarted","Data":"e23fb70929fab9a9cf572e4605b39b8f24c62dfa2121dc16e0c6fd5aedd03aad"} Jul 08 19:12:54.418916 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.418787 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" event={"ID":"20e87c68-5d08-4860-b03a-8d4cc3fdcbfb","Type":"ContainerStarted","Data":"172e4e30ddb4cb90c4f4446f4b1f573648c2fa8982b4bef9c071c2eee6982c8a"} Jul 08 19:12:54.420398 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.420359 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" event={"ID":"9e82e9f4-ccde-4b94-91f9-c5ed181ae83c","Type":"ContainerStarted","Data":"40aac54f7f5e3c567cd402537b85c5eb9f714ab33e027f262ec42bd332122c6c"} Jul 08 19:12:54.435571 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.435521 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-7ff64" podStartSLOduration=1.850610751 podStartE2EDuration="6.435503194s" totalImagesPullingTime="4.584892443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:54.433334163 +0000 UTC m=+83.102585851" watchObservedRunningTime="2026-07-08 19:12:54.435503194 +0000 UTC m=+83.104754885" Jul 08 19:12:54.451967 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.451916 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-674c95dc5c-vd42b" podStartSLOduration=1.842954529 podStartE2EDuration="6.451900615s" totalImagesPullingTime="4.608946086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:54.450470948 +0000 UTC m=+83.119722642" watchObservedRunningTime="2026-07-08 19:12:54.451900615 +0000 UTC m=+83.121152306" Jul 08 19:12:54.471307 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.471216 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-w9db9" podStartSLOduration=34.459308791 podStartE2EDuration="39.471195467s" totalImagesPullingTime="5.011886676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:54.470573923 +0000 UTC m=+83.139825613" watchObservedRunningTime="2026-07-08 19:12:54.471195467 +0000 UTC m=+83.140447159" Jul 08 19:12:54.541322 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.541286 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:54.597791 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.597756 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:54.881986 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.881921 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:54.976503 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:54.976471 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:55.757148 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:55.757114 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:55.817660 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:55.817624 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:56.381402 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:56.381355 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7bqln" Jul 08 19:12:58.466681 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:58.466632 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:58.519753 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:58.519723 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:58.657418 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:58.657391 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:12:58.917696 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:58.917665 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:12:58.973704 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:12:58.973662 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:01.672142 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:01.672108 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:01.735296 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:01.735266 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:02.877468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:02.877436 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:02.972024 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:02.971993 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:03.072883 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:03.072851 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:03.310082 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:03.310018 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:03.382653 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:03.382621 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:03.439811 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:03.439780 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:05.904985 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:05.904950 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:05.986977 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:05.986947 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:06.888042 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:06.888009 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:06.958959 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:06.958928 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:07.461065 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.461010 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:07.559218 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.559161 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:07.692804 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.692770 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:07.802651 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.802621 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vkmx9"] Jul 08 19:13:07.805861 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.805844 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:07.809184 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.809158 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 19:13:07.809349 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.809308 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pt55g\"" Jul 08 19:13:07.809505 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.809387 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 19:13:07.809572 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.809520 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 19:13:07.815511 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.815494 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 19:13:07.834057 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.834032 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:07.912186 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.912153 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-accelerators-collector-config\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:07.912360 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.912205 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:07.912360 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.912277 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-sys\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:07.912360 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.912314 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg4zc\" (UniqueName: \"kubernetes.io/projected/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-kube-api-access-rg4zc\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:07.912360 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.912343 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-tls\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:07.912557 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.912364 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-metrics-client-ca\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:07.912557 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.912420 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-textfile\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:07.912557 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.912438 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-wtmp\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:07.912557 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.912463 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-root\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:07.918001 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:07.917980 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:08.013365 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.013332 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-accelerators-collector-config\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.013806 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.013393 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.013806 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.013446 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-sys\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.013806 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.013479 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg4zc\" (UniqueName: \"kubernetes.io/projected/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-kube-api-access-rg4zc\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.013806 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.013545 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-sys\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.013806 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.013594 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-tls\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.013806 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.013625 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-metrics-client-ca\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.013806 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.013665 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-textfile\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.013806 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.013689 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-wtmp\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.013806 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.013718 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-root\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.013806 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.013812 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-root\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.014255 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.013926 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-wtmp\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.014255 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.013986 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-accelerators-collector-config\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.014255 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.014076 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-textfile\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.014409 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.014315 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-metrics-client-ca\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.016144 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.016123 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.016256 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.016181 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:08.016362 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.016340 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-node-exporter-tls\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.021919 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.021898 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg4zc\" (UniqueName: \"kubernetes.io/projected/aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a-kube-api-access-rg4zc\") pod \"node-exporter-vkmx9\" (UID: \"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a\") " pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.116046 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.115969 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vkmx9" Jul 08 19:13:08.117277 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.117248 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:08.127103 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:13:08.127078 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa8b384e_93cf_4e85_9e4b_0f0b3ff1724a.slice/crio-c159da5271ad0d029cdcac7637a53f90138248230ac913bed742e7bb7c071187 WatchSource:0}: Error finding container c159da5271ad0d029cdcac7637a53f90138248230ac913bed742e7bb7c071187: Status 404 returned error can't find the container with id c159da5271ad0d029cdcac7637a53f90138248230ac913bed742e7bb7c071187 Jul 08 19:13:08.221437 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.221399 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:08.321627 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.321587 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:08.472585 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.472498 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vkmx9" event={"ID":"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a","Type":"ContainerStarted","Data":"c159da5271ad0d029cdcac7637a53f90138248230ac913bed742e7bb7c071187"} Jul 08 19:13:08.735679 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.735094 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:08.820977 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.820772 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:08.939435 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:08.939352 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:09.039117 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:09.039086 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:09.342631 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:09.342601 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xg866" Jul 08 19:13:09.478190 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:09.478160 2563 generic.go:356] "Generic (PLEG): container finished" podID="aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a" containerID="f76e2859bb6a9398d7b8e456c08c9e8ffffc335f4645c12afefc67abb7aa091a" exitCode=0 Jul 08 19:13:09.478342 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:09.478196 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vkmx9" event={"ID":"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a","Type":"ContainerDied","Data":"f76e2859bb6a9398d7b8e456c08c9e8ffffc335f4645c12afefc67abb7aa091a"} Jul 08 19:13:10.403896 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:10.403868 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64b964b7cc-vpq7c" Jul 08 19:13:10.484504 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:10.484471 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vkmx9" event={"ID":"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a","Type":"ContainerStarted","Data":"49a05754f9a6bb66f481b93e742b5a3b698ea0ab4d86938918fc408086345884"} Jul 08 19:13:10.484661 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:10.484523 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vkmx9" event={"ID":"aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a","Type":"ContainerStarted","Data":"e496cd95110572c12d980379693fbf6eced4700d6584e1f8a9b6e499e57b6d24"} Jul 08 19:13:10.506349 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:10.506297 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vkmx9" podStartSLOduration=2.363003952 podStartE2EDuration="3.506281696s" totalImagesPullingTime="1.079403908s" totalInitContainerRuntime="63.873836ms" isStatefulPod=true podCreationTimestamp="2026-07-08 19:13:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:13:10.505051592 +0000 UTC m=+99.174303283" watchObservedRunningTime="2026-07-08 19:13:10.506281696 +0000 UTC m=+99.175533387" Jul 08 19:13:10.545327 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:10.545298 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:10.644957 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:10.644923 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:10.741657 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:10.741573 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:10.838275 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:10.838239 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:11.075520 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:11.075483 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:11.175872 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:11.175841 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:11.673507 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:11.673473 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:11.774680 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:11.774646 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:12.482946 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:12.482908 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:12.550764 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:12.550734 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:12.750682 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:12.750598 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:13.680152 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.680115 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" podUID="f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" containerName="registry" containerID="cri-o://dc3dcf31878be0b7697db057fb6e699f423175ed66ece45290fd9594f4de931e" gracePeriod=30 Jul 08 19:13:13.697003 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.696974 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:13.923196 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.923173 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:13:13.961100 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.961020 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-trusted-ca\") pod \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " Jul 08 19:13:13.961100 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.961068 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls\") pod \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " Jul 08 19:13:13.961294 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.961101 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-installation-pull-secrets\") pod \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " Jul 08 19:13:13.961294 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.961127 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9dfb\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-kube-api-access-m9dfb\") pod \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " Jul 08 19:13:13.961294 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.961174 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-bound-sa-token\") pod \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " Jul 08 19:13:13.961294 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.961220 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-image-registry-private-configuration\") pod \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " Jul 08 19:13:13.961294 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.961283 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-certificates\") pod \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " Jul 08 19:13:13.961559 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.961309 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-ca-trust-extracted\") pod \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\" (UID: \"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7\") " Jul 08 19:13:13.961559 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.961512 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" (UID: "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:13:13.961811 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.961781 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" (UID: "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:13:13.963555 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.963533 2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-certificates\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:13:13.963667 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.963558 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-trusted-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:13:13.964112 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.964004 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" (UID: "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:13:13.964112 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.964012 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" (UID: "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:13:13.964389 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.964343 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" (UID: "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:13:13.964389 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.964361 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" (UID: "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:13:13.964735 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.964702 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-kube-api-access-m9dfb" (OuterVolumeSpecName: "kube-api-access-m9dfb") pod "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" (UID: "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7"). InnerVolumeSpecName "kube-api-access-m9dfb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:13:13.971790 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:13.971769 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" (UID: "f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:13:14.024094 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.024039 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:14.063936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.063908 2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-ca-trust-extracted\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:13:14.063936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.063937 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-registry-tls\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:13:14.064132 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.063953 2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-installation-pull-secrets\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:13:14.064132 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.063968 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9dfb\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-kube-api-access-m9dfb\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:13:14.064132 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.063981 2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-bound-sa-token\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:13:14.064132 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.063994 2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7-image-registry-private-configuration\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:13:14.501303 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.501268 2563 generic.go:356] "Generic (PLEG): container finished" podID="f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" containerID="dc3dcf31878be0b7697db057fb6e699f423175ed66ece45290fd9594f4de931e" exitCode=0 Jul 08 19:13:14.501481 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.501309 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" Jul 08 19:13:14.501481 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.501319 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" event={"ID":"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7","Type":"ContainerDied","Data":"dc3dcf31878be0b7697db057fb6e699f423175ed66ece45290fd9594f4de931e"} Jul 08 19:13:14.501481 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.501366 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f8fc8f87d-p5vqg" event={"ID":"f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7","Type":"ContainerDied","Data":"c79be78f103c17cb3ea52a6a669f41d26954358194415ee711697d725d1a86d6"} Jul 08 19:13:14.501481 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.501417 2563 scope.go:117] "RemoveContainer" containerID="dc3dcf31878be0b7697db057fb6e699f423175ed66ece45290fd9594f4de931e" Jul 08 19:13:14.510062 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.510043 2563 scope.go:117] "RemoveContainer" containerID="dc3dcf31878be0b7697db057fb6e699f423175ed66ece45290fd9594f4de931e" Jul 08 19:13:14.510340 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:13:14.510322 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc3dcf31878be0b7697db057fb6e699f423175ed66ece45290fd9594f4de931e\": container with ID starting with dc3dcf31878be0b7697db057fb6e699f423175ed66ece45290fd9594f4de931e not found: ID does not exist" containerID="dc3dcf31878be0b7697db057fb6e699f423175ed66ece45290fd9594f4de931e" Jul 08 19:13:14.510410 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.510349 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc3dcf31878be0b7697db057fb6e699f423175ed66ece45290fd9594f4de931e"} err="failed to get container status \"dc3dcf31878be0b7697db057fb6e699f423175ed66ece45290fd9594f4de931e\": rpc error: code = NotFound desc = could not find container \"dc3dcf31878be0b7697db057fb6e699f423175ed66ece45290fd9594f4de931e\": container with ID starting with dc3dcf31878be0b7697db057fb6e699f423175ed66ece45290fd9594f4de931e not found: ID does not exist" Jul 08 19:13:14.523487 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.523461 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f8fc8f87d-p5vqg"] Jul 08 19:13:14.527562 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:14.527542 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-f8fc8f87d-p5vqg"] Jul 08 19:13:15.783439 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:15.783384 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:15.872559 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:15.872523 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:15.902678 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:15.902650 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" path="/var/lib/kubelet/pods/f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7/volumes" Jul 08 19:13:18.481943 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:18.481895 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:18.581615 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:18.581574 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:18.646867 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:18.646837 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:18.948435 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:18.948387 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:19.014963 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:19.014919 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:19.145333 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:19.145296 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:19.245346 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:19.245239 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:19.543129 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:19.543098 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:19.599879 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:19.599839 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:26.671578 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:26.671532 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:26.770751 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:26.770712 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:29.989312 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:29.989274 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:30.049904 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:30.049867 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:30.109674 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:30.109637 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:34.967659 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:34.967621 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:35.035792 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:35.035757 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:13:42.604953 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:42.604920 2563 generic.go:356] "Generic (PLEG): container finished" podID="1538d1d4-a106-4aa3-9caa-153393cfe60c" containerID="28b3ed485771f4075d99d207b1c4219ced1004410a93d548f5f9b858582bfd74" exitCode=0 Jul 08 19:13:42.605327 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:42.604975 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" event={"ID":"1538d1d4-a106-4aa3-9caa-153393cfe60c","Type":"ContainerDied","Data":"28b3ed485771f4075d99d207b1c4219ced1004410a93d548f5f9b858582bfd74"} Jul 08 19:13:42.605327 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:42.605310 2563 scope.go:117] "RemoveContainer" containerID="28b3ed485771f4075d99d207b1c4219ced1004410a93d548f5f9b858582bfd74" Jul 08 19:13:43.610393 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:43.610346 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-2j8ds" event={"ID":"1538d1d4-a106-4aa3-9caa-153393cfe60c","Type":"ContainerStarted","Data":"7f870c00aee7945ce88a7742b5796399c9988ae002beec6e51d52bec038b9add"} Jul 08 19:13:47.626362 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:47.626277 2563 generic.go:356] "Generic (PLEG): container finished" podID="b42ce879-9228-489a-a7a8-d12970b3cd3a" containerID="bc421569955eb275fcf53495bcb3ee0197fd62012e5005d20e31a0dcad971685" exitCode=0 Jul 08 19:13:47.626362 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:47.626334 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" event={"ID":"b42ce879-9228-489a-a7a8-d12970b3cd3a","Type":"ContainerDied","Data":"bc421569955eb275fcf53495bcb3ee0197fd62012e5005d20e31a0dcad971685"} Jul 08 19:13:47.626792 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:47.626692 2563 scope.go:117] "RemoveContainer" containerID="bc421569955eb275fcf53495bcb3ee0197fd62012e5005d20e31a0dcad971685" Jul 08 19:13:48.632439 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:48.632406 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jmjx2" event={"ID":"b42ce879-9228-489a-a7a8-d12970b3cd3a","Type":"ContainerStarted","Data":"97486a4a87d030371794a9d2407bc5f6fa8025cc2e5daf98eb6c5e5eb13de343"} Jul 08 19:13:52.647285 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:52.647251 2563 generic.go:356] "Generic (PLEG): container finished" podID="c0e7870b-512b-4301-8ebc-3479c3ac23d3" containerID="419300872e910475ed4764f6f95a5e043c0b83b6265a717401d85bb4e08bbf4f" exitCode=0 Jul 08 19:13:52.647671 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:52.647326 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" event={"ID":"c0e7870b-512b-4301-8ebc-3479c3ac23d3","Type":"ContainerDied","Data":"419300872e910475ed4764f6f95a5e043c0b83b6265a717401d85bb4e08bbf4f"} Jul 08 19:13:52.647733 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:52.647719 2563 scope.go:117] "RemoveContainer" containerID="419300872e910475ed4764f6f95a5e043c0b83b6265a717401d85bb4e08bbf4f" Jul 08 19:13:53.652996 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:13:53.652965 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-rgz68" event={"ID":"c0e7870b-512b-4301-8ebc-3479c3ac23d3","Type":"ContainerStarted","Data":"fa4c22171c3de0e8321fd04dcce08d295bb40a07da07391b08d5810fc8b653cd"} Jul 08 19:14:23.948124 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:14:23.948016 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:14:24.007476 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:14:24.007442 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:14:35.746819 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:14:35.746781 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:14:35.809353 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:14:35.809320 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:14:41.680584 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:14:41.680549 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:14:41.782136 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:14:41.782107 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:14:41.843202 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:14:41.843169 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:14:46.280180 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:14:46.280146 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:14:46.382043 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:14:46.382013 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:14:46.441069 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:14:46.441042 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:15.663332 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:15.663298 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:15.758584 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:15.758546 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:23.521236 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:23.521195 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:39.633495 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:39.633458 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:42.359552 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:42.359467 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:43.350415 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:43.350366 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:44.374919 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.374885 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v44bv"] Jul 08 19:15:44.375290 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.375131 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" containerName="registry" Jul 08 19:15:44.375290 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.375142 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" containerName="registry" Jul 08 19:15:44.375290 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.375225 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7b1e1c1-67b3-4846-a7d8-e4e7b2f4d3f7" containerName="registry" Jul 08 19:15:44.376925 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.376909 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v44bv" Jul 08 19:15:44.379854 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.379832 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 08 19:15:44.380250 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.380231 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:15:44.380418 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.380400 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-gfmcn\"" Jul 08 19:15:44.390587 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.390567 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v44bv"] Jul 08 19:15:44.460499 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.460463 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-548k8\" (UniqueName: \"kubernetes.io/projected/e5e0fd14-9dad-4602-86c1-eccc80d56610-kube-api-access-548k8\") pod \"cert-manager-operator-controller-manager-55c444bf86-v44bv\" (UID: \"e5e0fd14-9dad-4602-86c1-eccc80d56610\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v44bv" Jul 08 19:15:44.460672 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.460528 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5e0fd14-9dad-4602-86c1-eccc80d56610-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v44bv\" (UID: \"e5e0fd14-9dad-4602-86c1-eccc80d56610\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v44bv" Jul 08 19:15:44.561396 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.561333 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-548k8\" (UniqueName: \"kubernetes.io/projected/e5e0fd14-9dad-4602-86c1-eccc80d56610-kube-api-access-548k8\") pod \"cert-manager-operator-controller-manager-55c444bf86-v44bv\" (UID: \"e5e0fd14-9dad-4602-86c1-eccc80d56610\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v44bv" Jul 08 19:15:44.561587 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.561448 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5e0fd14-9dad-4602-86c1-eccc80d56610-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v44bv\" (UID: \"e5e0fd14-9dad-4602-86c1-eccc80d56610\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v44bv" Jul 08 19:15:44.561841 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.561823 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5e0fd14-9dad-4602-86c1-eccc80d56610-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v44bv\" (UID: \"e5e0fd14-9dad-4602-86c1-eccc80d56610\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v44bv" Jul 08 19:15:44.572066 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.572040 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-548k8\" (UniqueName: \"kubernetes.io/projected/e5e0fd14-9dad-4602-86c1-eccc80d56610-kube-api-access-548k8\") pod \"cert-manager-operator-controller-manager-55c444bf86-v44bv\" (UID: \"e5e0fd14-9dad-4602-86c1-eccc80d56610\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v44bv" Jul 08 19:15:44.686529 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.686450 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v44bv" Jul 08 19:15:44.815132 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:44.815095 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v44bv"] Jul 08 19:15:44.819559 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:15:44.819527 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e0fd14_9dad_4602_86c1_eccc80d56610.slice/crio-3f48231a9b092fb959d480ec228f5145974cce5b2bcb850eea67c883e08cb390 WatchSource:0}: Error finding container 3f48231a9b092fb959d480ec228f5145974cce5b2bcb850eea67c883e08cb390: Status 404 returned error can't find the container with id 3f48231a9b092fb959d480ec228f5145974cce5b2bcb850eea67c883e08cb390 Jul 08 19:15:45.039328 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:45.039296 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v44bv" event={"ID":"e5e0fd14-9dad-4602-86c1-eccc80d56610","Type":"ContainerStarted","Data":"3f48231a9b092fb959d480ec228f5145974cce5b2bcb850eea67c883e08cb390"} Jul 08 19:15:45.978159 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:45.978127 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:46.078233 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:46.078195 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:47.928835 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:47.928781 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:48.030219 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:48.030175 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:48.055617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:48.055582 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v44bv" event={"ID":"e5e0fd14-9dad-4602-86c1-eccc80d56610","Type":"ContainerStarted","Data":"2e82e21485e4323ac92a86590edca0857b60f9e3c315bbee43af8af2434fbfa3"} Jul 08 19:15:48.078205 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:48.078155 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v44bv" podStartSLOduration=1.581180109 podStartE2EDuration="4.078141926s" totalImagesPullingTime="2.496961817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:15:48.076658435 +0000 UTC m=+256.745910138" watchObservedRunningTime="2026-07-08 19:15:48.078141926 +0000 UTC m=+256.747393617" Jul 08 19:15:48.500305 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:48.500266 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:48.604783 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:48.604749 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:50.132669 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:50.132630 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:50.316173 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:50.316141 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:52.962908 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:52.962829 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:53.018437 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:53.018400 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:56.723153 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:56.723103 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:56.826705 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:56.826667 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:15:58.330200 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:15:58.330163 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:01.376550 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:01.376513 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:01.477687 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:01.477650 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:07.144593 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:07.144546 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:07.245693 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:07.245660 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:08.285523 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:08.285490 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:09.184861 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:09.184830 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:10.769346 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:10.769311 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:10.827533 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:10.827498 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:21.985854 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:21.985826 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:22.075867 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.075836 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:22.583388 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.583321 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:22.807726 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.807667 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:22.939689 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.939613 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf"] Jul 08 19:16:22.942700 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.942683 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:22.945661 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.945636 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 19:16:22.945761 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.945705 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 19:16:22.946043 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.946029 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 19:16:22.946087 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.946044 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 19:16:22.946273 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.946260 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9h9dk\"" Jul 08 19:16:22.959966 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.959942 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c2b21b5-fba4-4572-b4f6-a900073ecdec-manifests\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:22.960075 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.959976 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c2b21b5-fba4-4572-b4f6-a900073ecdec-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:22.960075 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.960015 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c2b21b5-fba4-4572-b4f6-a900073ecdec-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:22.960179 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.960105 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c2b21b5-fba4-4572-b4f6-a900073ecdec-tmp\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:22.960179 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.960149 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6whk\" (UniqueName: \"kubernetes.io/projected/8c2b21b5-fba4-4572-b4f6-a900073ecdec-kube-api-access-b6whk\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:22.960321 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.960304 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf"] Jul 08 19:16:22.979203 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:22.979176 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:23.061477 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.061451 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c2b21b5-fba4-4572-b4f6-a900073ecdec-tmp\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:23.061857 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.061494 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6whk\" (UniqueName: \"kubernetes.io/projected/8c2b21b5-fba4-4572-b4f6-a900073ecdec-kube-api-access-b6whk\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:23.061857 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.061527 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c2b21b5-fba4-4572-b4f6-a900073ecdec-manifests\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:23.061857 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.061542 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c2b21b5-fba4-4572-b4f6-a900073ecdec-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:23.061857 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.061565 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c2b21b5-fba4-4572-b4f6-a900073ecdec-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:23.062049 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.061954 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c2b21b5-fba4-4572-b4f6-a900073ecdec-manifests\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:23.063720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.063700 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c2b21b5-fba4-4572-b4f6-a900073ecdec-tmp\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:23.063833 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.063815 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c2b21b5-fba4-4572-b4f6-a900073ecdec-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:23.063953 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.063935 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c2b21b5-fba4-4572-b4f6-a900073ecdec-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:23.075200 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.075174 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6whk\" (UniqueName: \"kubernetes.io/projected/8c2b21b5-fba4-4572-b4f6-a900073ecdec-kube-api-access-b6whk\") pod \"opendatahub-operator-controller-manager-7dc4d9889-h4phf\" (UID: \"8c2b21b5-fba4-4572-b4f6-a900073ecdec\") " pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:23.174955 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.174921 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:23.252739 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.252648 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:23.275581 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.275552 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:23.377580 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.377543 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:23.391981 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.391963 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf"] Jul 08 19:16:23.394440 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:16:23.394416 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c2b21b5_fba4_4572_b4f6_a900073ecdec.slice/crio-a30ccfd678dbc76dea8dd583fa4f107b778a572437737f2baa258283c8c9b13c WatchSource:0}: Error finding container a30ccfd678dbc76dea8dd583fa4f107b778a572437737f2baa258283c8c9b13c: Status 404 returned error can't find the container with id a30ccfd678dbc76dea8dd583fa4f107b778a572437737f2baa258283c8c9b13c Jul 08 19:16:23.475695 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.475667 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:23.577162 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.577122 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:23.676041 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:23.676001 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:24.206326 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:24.206288 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" event={"ID":"8c2b21b5-fba4-4572-b4f6-a900073ecdec","Type":"ContainerStarted","Data":"a30ccfd678dbc76dea8dd583fa4f107b778a572437737f2baa258283c8c9b13c"} Jul 08 19:16:27.219937 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:27.219903 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" event={"ID":"8c2b21b5-fba4-4572-b4f6-a900073ecdec","Type":"ContainerStarted","Data":"77e1f61115582f4801ce481a124b86015f313a19561c52980b34ea7f5a0818ec"} Jul 08 19:16:28.225114 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:28.225080 2563 generic.go:356] "Generic (PLEG): container finished" podID="8c2b21b5-fba4-4572-b4f6-a900073ecdec" containerID="77e1f61115582f4801ce481a124b86015f313a19561c52980b34ea7f5a0818ec" exitCode=0 Jul 08 19:16:28.225566 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:28.225164 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" event={"ID":"8c2b21b5-fba4-4572-b4f6-a900073ecdec","Type":"ContainerDied","Data":"77e1f61115582f4801ce481a124b86015f313a19561c52980b34ea7f5a0818ec"} Jul 08 19:16:29.230801 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:29.230771 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" event={"ID":"8c2b21b5-fba4-4572-b4f6-a900073ecdec","Type":"ContainerStarted","Data":"0472c441fa7ccfb826417cf97d0cd44b9234d97c1aa0a00e775693687127e1f3"} Jul 08 19:16:29.231181 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:29.230884 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:29.254970 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:29.254853 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" podStartSLOduration=2.83452805 podStartE2EDuration="7.254833038s" totalImagesPullingTime="3.378740047s" totalInitContainerRuntime="1.041564941s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:16:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:16:29.253102497 +0000 UTC m=+297.922354187" watchObservedRunningTime="2026-07-08 19:16:29.254833038 +0000 UTC m=+297.924084729" Jul 08 19:16:31.509204 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:31.509171 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:31.614065 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:31.614031 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:31.722802 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:31.722775 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:16:31.727049 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:31.727026 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:16:31.734920 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:31.734898 2563 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 19:16:34.767517 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:34.767477 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:34.877076 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:34.877033 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:40.237007 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:40.236978 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7dc4d9889-h4phf" Jul 08 19:16:42.148707 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.148668 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:42.282215 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.282179 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:42.311764 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.311733 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-f57846485-vscr4"] Jul 08 19:16:42.313729 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.313701 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:42.318114 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.318096 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:16:42.318332 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.318318 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 19:16:42.319144 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.319122 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 19:16:42.319253 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.319164 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 19:16:42.319633 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.319617 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5jpp6\"" Jul 08 19:16:42.320492 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.320474 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 19:16:42.406174 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.406092 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f57846485-vscr4"] Jul 08 19:16:42.428194 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.428162 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38c63e5f-4aaf-41b7-afb5-99e804356d87-cert\") pod \"lws-controller-manager-f57846485-vscr4\" (UID: \"38c63e5f-4aaf-41b7-afb5-99e804356d87\") " pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:42.428362 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.428200 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/38c63e5f-4aaf-41b7-afb5-99e804356d87-metrics-cert\") pod \"lws-controller-manager-f57846485-vscr4\" (UID: \"38c63e5f-4aaf-41b7-afb5-99e804356d87\") " pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:42.428362 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.428240 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz8p6\" (UniqueName: \"kubernetes.io/projected/38c63e5f-4aaf-41b7-afb5-99e804356d87-kube-api-access-lz8p6\") pod \"lws-controller-manager-f57846485-vscr4\" (UID: \"38c63e5f-4aaf-41b7-afb5-99e804356d87\") " pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:42.428362 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.428276 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/38c63e5f-4aaf-41b7-afb5-99e804356d87-manager-config\") pod \"lws-controller-manager-f57846485-vscr4\" (UID: \"38c63e5f-4aaf-41b7-afb5-99e804356d87\") " pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:42.529134 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.529086 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/38c63e5f-4aaf-41b7-afb5-99e804356d87-manager-config\") pod \"lws-controller-manager-f57846485-vscr4\" (UID: \"38c63e5f-4aaf-41b7-afb5-99e804356d87\") " pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:42.529302 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.529181 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38c63e5f-4aaf-41b7-afb5-99e804356d87-cert\") pod \"lws-controller-manager-f57846485-vscr4\" (UID: \"38c63e5f-4aaf-41b7-afb5-99e804356d87\") " pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:42.529302 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.529213 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/38c63e5f-4aaf-41b7-afb5-99e804356d87-metrics-cert\") pod \"lws-controller-manager-f57846485-vscr4\" (UID: \"38c63e5f-4aaf-41b7-afb5-99e804356d87\") " pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:42.529436 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.529357 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz8p6\" (UniqueName: \"kubernetes.io/projected/38c63e5f-4aaf-41b7-afb5-99e804356d87-kube-api-access-lz8p6\") pod \"lws-controller-manager-f57846485-vscr4\" (UID: \"38c63e5f-4aaf-41b7-afb5-99e804356d87\") " pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:42.529880 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.529852 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/38c63e5f-4aaf-41b7-afb5-99e804356d87-manager-config\") pod \"lws-controller-manager-f57846485-vscr4\" (UID: \"38c63e5f-4aaf-41b7-afb5-99e804356d87\") " pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:42.532424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.532399 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38c63e5f-4aaf-41b7-afb5-99e804356d87-cert\") pod \"lws-controller-manager-f57846485-vscr4\" (UID: \"38c63e5f-4aaf-41b7-afb5-99e804356d87\") " pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:42.532542 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.532404 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/38c63e5f-4aaf-41b7-afb5-99e804356d87-metrics-cert\") pod \"lws-controller-manager-f57846485-vscr4\" (UID: \"38c63e5f-4aaf-41b7-afb5-99e804356d87\") " pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:42.553029 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.553003 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz8p6\" (UniqueName: \"kubernetes.io/projected/38c63e5f-4aaf-41b7-afb5-99e804356d87-kube-api-access-lz8p6\") pod \"lws-controller-manager-f57846485-vscr4\" (UID: \"38c63e5f-4aaf-41b7-afb5-99e804356d87\") " pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:42.622542 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.622513 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:42.647106 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.647074 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:42.774045 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.774010 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f57846485-vscr4"] Jul 08 19:16:42.779897 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:16:42.779874 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38c63e5f_4aaf_41b7_afb5_99e804356d87.slice/crio-aedda3ee128fab98d9540b14cd56d4454f25853efb56a1a742c37fcdc8752889 WatchSource:0}: Error finding container aedda3ee128fab98d9540b14cd56d4454f25853efb56a1a742c37fcdc8752889: Status 404 returned error can't find the container with id aedda3ee128fab98d9540b14cd56d4454f25853efb56a1a742c37fcdc8752889 Jul 08 19:16:42.781860 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.781843 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:16:42.856979 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:42.856943 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:43.126815 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:43.126784 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:43.216623 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:43.216586 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:43.285164 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:43.285130 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" event={"ID":"38c63e5f-4aaf-41b7-afb5-99e804356d87","Type":"ContainerStarted","Data":"aedda3ee128fab98d9540b14cd56d4454f25853efb56a1a742c37fcdc8752889"} Jul 08 19:16:43.342330 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:43.342295 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:43.477297 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:43.477206 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:43.572429 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:43.572395 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:44.186130 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:44.186089 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:44.408413 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:44.408358 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:44.479020 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:44.478926 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:45.400596 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:45.400560 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:46.298711 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:46.298667 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" event={"ID":"38c63e5f-4aaf-41b7-afb5-99e804356d87","Type":"ContainerStarted","Data":"62f8e9e7e4559ee288c66bb1baa076b7801d7d0a7dafa8692c1a536449593969"} Jul 08 19:16:46.299082 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:46.298783 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:46.316093 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:46.316043 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" podStartSLOduration=1.832023994 podStartE2EDuration="4.316026029s" totalImagesPullingTime="2.484002035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:16:46.315674892 +0000 UTC m=+314.984926583" watchObservedRunningTime="2026-07-08 19:16:46.316026029 +0000 UTC m=+314.985277722" Jul 08 19:16:50.108264 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.108230 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:50.416745 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.416664 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp"] Jul 08 19:16:50.418957 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.418941 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.422649 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.422630 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 19:16:50.422801 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.422763 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 19:16:50.425188 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.425173 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 19:16:50.428114 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.428098 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-4flgz\"" Jul 08 19:16:50.461325 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.461293 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp"] Jul 08 19:16:50.500185 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.500146 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.500185 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.500191 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.500467 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.500209 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.500467 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.500255 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49w49\" (UniqueName: \"kubernetes.io/projected/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-kube-api-access-49w49\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.500467 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.500315 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.500467 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.500396 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.500467 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.500450 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.500648 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.500482 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.500648 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.500514 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.601920 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.601886 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.601920 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.601926 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.602141 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.601947 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.602141 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.601971 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49w49\" (UniqueName: \"kubernetes.io/projected/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-kube-api-access-49w49\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.602141 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.602033 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.602141 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.602064 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.602141 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.602099 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.602141 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.602133 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.602471 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.602164 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.602471 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.602449 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.602589 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.602486 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.602706 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.602684 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.602776 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.602708 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.602968 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.602947 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.604422 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.604403 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.604552 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.604533 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.618547 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.618522 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49w49\" (UniqueName: \"kubernetes.io/projected/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-kube-api-access-49w49\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.619713 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.619693 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.729607 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.729524 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:50.861698 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:50.861673 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp"] Jul 08 19:16:50.864217 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:16:50.864190 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c8928c5_d62e_48d2_aebd_2f9c34b63c40.slice/crio-a3e36c5c8a4266f57e3d2496e2799e321d57e7dbc39fe2dd19817b54157c621e WatchSource:0}: Error finding container a3e36c5c8a4266f57e3d2496e2799e321d57e7dbc39fe2dd19817b54157c621e: Status 404 returned error can't find the container with id a3e36c5c8a4266f57e3d2496e2799e321d57e7dbc39fe2dd19817b54157c621e Jul 08 19:16:51.018017 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.017985 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn"] Jul 08 19:16:51.020568 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.020553 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.026292 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.026273 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 19:16:51.026485 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.026466 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 19:16:51.026567 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.026548 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 19:16:51.026632 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.026555 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-db88q\"" Jul 08 19:16:51.062275 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.062213 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn"] Jul 08 19:16:51.106908 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.106876 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0fbfa103-5cdd-4421-9061-f26746bc71a6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.107042 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.106914 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0fbfa103-5cdd-4421-9061-f26746bc71a6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.107042 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.107019 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0fbfa103-5cdd-4421-9061-f26746bc71a6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.107117 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.107056 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dr9j\" (UniqueName: \"kubernetes.io/projected/0fbfa103-5cdd-4421-9061-f26746bc71a6-kube-api-access-5dr9j\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.107117 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.107082 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fbfa103-5cdd-4421-9061-f26746bc71a6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.107117 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.107102 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0fbfa103-5cdd-4421-9061-f26746bc71a6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.107205 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.107144 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fbfa103-5cdd-4421-9061-f26746bc71a6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.207897 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.207864 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0fbfa103-5cdd-4421-9061-f26746bc71a6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.208308 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.207905 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dr9j\" (UniqueName: \"kubernetes.io/projected/0fbfa103-5cdd-4421-9061-f26746bc71a6-kube-api-access-5dr9j\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.208308 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.207924 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fbfa103-5cdd-4421-9061-f26746bc71a6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.208308 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.207939 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0fbfa103-5cdd-4421-9061-f26746bc71a6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.208308 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.207978 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fbfa103-5cdd-4421-9061-f26746bc71a6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.208308 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.208107 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0fbfa103-5cdd-4421-9061-f26746bc71a6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.208308 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.208145 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0fbfa103-5cdd-4421-9061-f26746bc71a6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.208690 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.208662 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0fbfa103-5cdd-4421-9061-f26746bc71a6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.210185 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.210165 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fbfa103-5cdd-4421-9061-f26746bc71a6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.210393 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.210357 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0fbfa103-5cdd-4421-9061-f26746bc71a6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.210445 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.210401 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0fbfa103-5cdd-4421-9061-f26746bc71a6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.210488 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.210451 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0fbfa103-5cdd-4421-9061-f26746bc71a6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.217063 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.217043 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fbfa103-5cdd-4421-9061-f26746bc71a6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.217293 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.217278 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dr9j\" (UniqueName: \"kubernetes.io/projected/0fbfa103-5cdd-4421-9061-f26746bc71a6-kube-api-access-5dr9j\") pod \"istiod-openshift-gateway-94bb8fbfd-h89hn\" (UID: \"0fbfa103-5cdd-4421-9061-f26746bc71a6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.320047 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.319960 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" event={"ID":"4c8928c5-d62e-48d2-aebd-2f9c34b63c40","Type":"ContainerStarted","Data":"a3e36c5c8a4266f57e3d2496e2799e321d57e7dbc39fe2dd19817b54157c621e"} Jul 08 19:16:51.330280 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.330248 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:51.460809 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.460776 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn"] Jul 08 19:16:51.463601 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:16:51.463575 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fbfa103_5cdd_4421_9061_f26746bc71a6.slice/crio-c0ea4f2ae7dcab1902f036c2ceec40516e8896498e3e621544e34422a67c0397 WatchSource:0}: Error finding container c0ea4f2ae7dcab1902f036c2ceec40516e8896498e3e621544e34422a67c0397: Status 404 returned error can't find the container with id c0ea4f2ae7dcab1902f036c2ceec40516e8896498e3e621544e34422a67c0397 Jul 08 19:16:51.821256 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:51.821221 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:52.327297 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:52.327251 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" event={"ID":"0fbfa103-5cdd-4421-9061-f26746bc71a6","Type":"ContainerStarted","Data":"c0ea4f2ae7dcab1902f036c2ceec40516e8896498e3e621544e34422a67c0397"} Jul 08 19:16:54.010735 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:54.010688 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:54.170922 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:54.170882 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:54.261622 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:54.261525 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:54.366658 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:54.366610 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:54.466140 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:54.466101 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:54.575809 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:54.575731 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:16:54.612965 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:54.612911 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:16:54.613027 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:54.613013 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:16:54.613062 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:54.613045 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:16:54.614155 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:54.614125 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:16:54.614271 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:54.614189 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:16:55.341730 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:55.341691 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" event={"ID":"4c8928c5-d62e-48d2-aebd-2f9c34b63c40","Type":"ContainerStarted","Data":"d0e1f0f1e21f634b78eb5f83ed6e7387848b1373c8e2aa7cf3e250d6df53b77e"} Jul 08 19:16:55.343260 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:55.343227 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" event={"ID":"0fbfa103-5cdd-4421-9061-f26746bc71a6","Type":"ContainerStarted","Data":"f6544df422ee8b335ce0afa63ec3bbc7219cd720969aafc5f704c168f8ab01cf"} Jul 08 19:16:55.343390 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:55.343360 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:55.345065 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:55.345042 2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-h89hn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 19:16:55.345157 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:55.345084 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" podUID="0fbfa103-5cdd-4421-9061-f26746bc71a6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 19:16:55.383520 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:55.383464 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" podStartSLOduration=1.636959828 podStartE2EDuration="5.383450361s" totalImagesPullingTime="3.746490533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:16:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:16:55.382151422 +0000 UTC m=+324.051403127" watchObservedRunningTime="2026-07-08 19:16:55.383450361 +0000 UTC m=+324.052702052" Jul 08 19:16:55.420242 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:55.420199 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" podStartSLOduration=2.271971566 podStartE2EDuration="5.420182504s" totalImagesPullingTime="3.148210938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:16:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:16:55.419678404 +0000 UTC m=+324.088930093" watchObservedRunningTime="2026-07-08 19:16:55.420182504 +0000 UTC m=+324.089434195" Jul 08 19:16:55.730438 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:55.730331 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:16:55.732101 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:55.732071 2563 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Jul 08 19:16:55.732324 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:55.732295 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" podUID="4c8928c5-d62e-48d2-aebd-2f9c34b63c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Jul 08 19:16:56.348395 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:56.348357 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h89hn" Jul 08 19:16:56.730832 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:56.730750 2563 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Jul 08 19:16:56.730832 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:56.730803 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" podUID="4c8928c5-d62e-48d2-aebd-2f9c34b63c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Jul 08 19:16:56.819073 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:56.819037 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp"] Jul 08 19:16:57.304780 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:57.304752 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-f57846485-vscr4" Jul 08 19:16:57.730906 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:57.730824 2563 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Jul 08 19:16:57.730906 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:57.730876 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" podUID="4c8928c5-d62e-48d2-aebd-2f9c34b63c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Jul 08 19:16:58.355911 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:16:58.355871 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" podUID="4c8928c5-d62e-48d2-aebd-2f9c34b63c40" containerName="istio-proxy" containerID="cri-o://d0e1f0f1e21f634b78eb5f83ed6e7387848b1373c8e2aa7cf3e250d6df53b77e" gracePeriod=30 Jul 08 19:17:03.591994 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.591967 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:17:03.715585 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.715510 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-token\") pod \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " Jul 08 19:17:03.715585 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.715563 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istiod-ca-cert\") pod \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " Jul 08 19:17:03.715803 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.715593 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-data\") pod \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " Jul 08 19:17:03.715803 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.715610 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-envoy\") pod \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " Jul 08 19:17:03.715803 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.715637 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-podinfo\") pod \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " Jul 08 19:17:03.715803 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.715686 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49w49\" (UniqueName: \"kubernetes.io/projected/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-kube-api-access-49w49\") pod \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " Jul 08 19:17:03.715803 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.715712 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-workload-socket\") pod \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " Jul 08 19:17:03.715803 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.715753 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-workload-certs\") pod \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " Jul 08 19:17:03.715803 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.715782 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-credential-socket\") pod \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\" (UID: \"4c8928c5-d62e-48d2-aebd-2f9c34b63c40\") " Jul 08 19:17:03.716130 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.715926 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-data" (OuterVolumeSpecName: "istio-data") pod "4c8928c5-d62e-48d2-aebd-2f9c34b63c40" (UID: "4c8928c5-d62e-48d2-aebd-2f9c34b63c40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:17:03.716130 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.716030 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4c8928c5-d62e-48d2-aebd-2f9c34b63c40" (UID: "4c8928c5-d62e-48d2-aebd-2f9c34b63c40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:17:03.716130 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.716121 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4c8928c5-d62e-48d2-aebd-2f9c34b63c40" (UID: "4c8928c5-d62e-48d2-aebd-2f9c34b63c40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:17:03.716268 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.716184 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istiod-ca-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:17:03.716268 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.716193 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4c8928c5-d62e-48d2-aebd-2f9c34b63c40" (UID: "4c8928c5-d62e-48d2-aebd-2f9c34b63c40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:17:03.716268 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.716204 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-data\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:17:03.716422 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.716307 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4c8928c5-d62e-48d2-aebd-2f9c34b63c40" (UID: "4c8928c5-d62e-48d2-aebd-2f9c34b63c40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:17:03.718390 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.718352 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-token" (OuterVolumeSpecName: "istio-token") pod "4c8928c5-d62e-48d2-aebd-2f9c34b63c40" (UID: "4c8928c5-d62e-48d2-aebd-2f9c34b63c40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:17:03.718526 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.718353 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4c8928c5-d62e-48d2-aebd-2f9c34b63c40" (UID: "4c8928c5-d62e-48d2-aebd-2f9c34b63c40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:17:03.718526 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.718424 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4c8928c5-d62e-48d2-aebd-2f9c34b63c40" (UID: "4c8928c5-d62e-48d2-aebd-2f9c34b63c40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:17:03.718526 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.718496 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-kube-api-access-49w49" (OuterVolumeSpecName: "kube-api-access-49w49") pod "4c8928c5-d62e-48d2-aebd-2f9c34b63c40" (UID: "4c8928c5-d62e-48d2-aebd-2f9c34b63c40"). InnerVolumeSpecName "kube-api-access-49w49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:17:03.817207 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.817165 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49w49\" (UniqueName: \"kubernetes.io/projected/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-kube-api-access-49w49\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:17:03.817207 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.817199 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-workload-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:17:03.817207 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.817209 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-workload-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:17:03.817207 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.817218 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-credential-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:17:03.817511 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.817226 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-token\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:17:03.817511 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.817234 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-envoy\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:17:03.817511 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:03.817242 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c8928c5-d62e-48d2-aebd-2f9c34b63c40-istio-podinfo\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:17:04.381874 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:04.381835 2563 generic.go:356] "Generic (PLEG): container finished" podID="4c8928c5-d62e-48d2-aebd-2f9c34b63c40" containerID="d0e1f0f1e21f634b78eb5f83ed6e7387848b1373c8e2aa7cf3e250d6df53b77e" exitCode=0 Jul 08 19:17:04.382063 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:04.381878 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" Jul 08 19:17:04.382063 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:04.381881 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" event={"ID":"4c8928c5-d62e-48d2-aebd-2f9c34b63c40","Type":"ContainerDied","Data":"d0e1f0f1e21f634b78eb5f83ed6e7387848b1373c8e2aa7cf3e250d6df53b77e"} Jul 08 19:17:04.382063 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:04.381935 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp" event={"ID":"4c8928c5-d62e-48d2-aebd-2f9c34b63c40","Type":"ContainerDied","Data":"a3e36c5c8a4266f57e3d2496e2799e321d57e7dbc39fe2dd19817b54157c621e"} Jul 08 19:17:04.382063 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:04.381963 2563 scope.go:117] "RemoveContainer" containerID="d0e1f0f1e21f634b78eb5f83ed6e7387848b1373c8e2aa7cf3e250d6df53b77e" Jul 08 19:17:04.390085 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:04.390069 2563 scope.go:117] "RemoveContainer" containerID="d0e1f0f1e21f634b78eb5f83ed6e7387848b1373c8e2aa7cf3e250d6df53b77e" Jul 08 19:17:04.390328 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:17:04.390309 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0e1f0f1e21f634b78eb5f83ed6e7387848b1373c8e2aa7cf3e250d6df53b77e\": container with ID starting with d0e1f0f1e21f634b78eb5f83ed6e7387848b1373c8e2aa7cf3e250d6df53b77e not found: ID does not exist" containerID="d0e1f0f1e21f634b78eb5f83ed6e7387848b1373c8e2aa7cf3e250d6df53b77e" Jul 08 19:17:04.390403 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:04.390337 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e1f0f1e21f634b78eb5f83ed6e7387848b1373c8e2aa7cf3e250d6df53b77e"} err="failed to get container status \"d0e1f0f1e21f634b78eb5f83ed6e7387848b1373c8e2aa7cf3e250d6df53b77e\": rpc error: code = NotFound desc = could not find container \"d0e1f0f1e21f634b78eb5f83ed6e7387848b1373c8e2aa7cf3e250d6df53b77e\": container with ID starting with d0e1f0f1e21f634b78eb5f83ed6e7387848b1373c8e2aa7cf3e250d6df53b77e not found: ID does not exist" Jul 08 19:17:04.401444 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:04.401421 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp"] Jul 08 19:17:04.404023 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:04.404002 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-sfgmp"] Jul 08 19:17:05.169261 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:05.169223 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:17:05.231127 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:05.231092 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:17:05.903113 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:05.903067 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c8928c5-d62e-48d2-aebd-2f9c34b63c40" path="/var/lib/kubelet/pods/4c8928c5-d62e-48d2-aebd-2f9c34b63c40/volumes" Jul 08 19:17:06.912515 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:06.912480 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:17:06.969163 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:06.969134 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:17:43.675282 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:43.675243 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:17:43.770914 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:43.770883 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:17:43.852459 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:43.852422 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:17:44.620829 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:44.620795 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:17:44.807863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:44.807827 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:17:45.451249 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:17:45.451205 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:12.879536 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:12.879480 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:16.219310 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:16.219270 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:16.902025 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:16.901969 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:17.888930 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:17.888898 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:22.091248 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:22.091210 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:27.110658 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:27.110588 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:32.597424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:32.597361 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:32.697200 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:32.697164 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:36.292644 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:36.292605 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:37.162273 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:37.162237 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:38.557897 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:38.557867 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:38.619515 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:38.619482 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:43.480904 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:43.480870 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:43.664275 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:43.664238 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:43.793349 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:43.793317 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:44.723299 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:44.723264 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:46.338952 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:46.338909 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:47.980290 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:47.980250 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:49.518457 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:49.518412 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:49.637690 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:49.637645 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:54.301904 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:54.301826 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:54.403519 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:54.403466 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:58.119693 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:58.119658 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:58.197037 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:58.197000 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:58.306276 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:58.306243 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:18:58.405866 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:18:58.405784 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:02.047719 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:02.047681 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:02.146014 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:02.145981 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:08.525904 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:08.525871 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:08.596028 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:08.595998 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:12.879917 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:12.879879 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:12.999865 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:12.999829 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:16.681066 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:16.681021 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:16.777064 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:16.777029 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:16.877011 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:16.876967 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:16.949651 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:16.949566 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:22.079522 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:22.079493 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:22.180424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:22.180365 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:30.274566 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:30.274487 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:30.374932 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:30.374894 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:39.870989 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.870957 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds"] Jul 08 19:19:39.871472 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.871356 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c8928c5-d62e-48d2-aebd-2f9c34b63c40" containerName="istio-proxy" Jul 08 19:19:39.871472 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.871385 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c8928c5-d62e-48d2-aebd-2f9c34b63c40" containerName="istio-proxy" Jul 08 19:19:39.871574 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.871478 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c8928c5-d62e-48d2-aebd-2f9c34b63c40" containerName="istio-proxy" Jul 08 19:19:39.874552 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.874532 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:39.878045 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.878020 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-bpzlm\"" Jul 08 19:19:39.887855 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.887832 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds"] Jul 08 19:19:39.902246 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.902218 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:39.914502 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.914477 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:39.914634 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.914533 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:39.914634 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.914575 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:39.914634 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.914601 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1890d3fc-58bc-446b-b53f-300a7b637bab-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:39.914797 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.914643 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:39.914797 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.914685 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1890d3fc-58bc-446b-b53f-300a7b637bab-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:39.914797 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.914754 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:39.914797 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.914780 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1890d3fc-58bc-446b-b53f-300a7b637bab-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:39.914953 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:39.914855 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46rrv\" (UniqueName: \"kubernetes.io/projected/1890d3fc-58bc-446b-b53f-300a7b637bab-kube-api-access-46rrv\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.011087 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.011051 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:40.015219 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.015193 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.015357 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.015225 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1890d3fc-58bc-446b-b53f-300a7b637bab-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.015357 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.015255 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46rrv\" (UniqueName: \"kubernetes.io/projected/1890d3fc-58bc-446b-b53f-300a7b637bab-kube-api-access-46rrv\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.015357 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.015274 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.015357 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.015310 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.015357 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.015330 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.015357 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.015345 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1890d3fc-58bc-446b-b53f-300a7b637bab-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.015709 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.015367 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.015709 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.015431 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1890d3fc-58bc-446b-b53f-300a7b637bab-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.015858 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.015837 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.015911 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.015879 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.015965 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.015912 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.016021 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.015979 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.016094 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.016077 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1890d3fc-58bc-446b-b53f-300a7b637bab-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.017721 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.017693 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1890d3fc-58bc-446b-b53f-300a7b637bab-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.017971 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.017952 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1890d3fc-58bc-446b-b53f-300a7b637bab-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.023461 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.023442 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46rrv\" (UniqueName: \"kubernetes.io/projected/1890d3fc-58bc-446b-b53f-300a7b637bab-kube-api-access-46rrv\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.023583 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.023562 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1890d3fc-58bc-446b-b53f-300a7b637bab-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xg7ds\" (UID: \"1890d3fc-58bc-446b-b53f-300a7b637bab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.185001 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.184907 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:40.320331 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.320304 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds"] Jul 08 19:19:40.322426 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:19:40.322395 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1890d3fc_58bc_446b_b53f_300a7b637bab.slice/crio-57262c96f827a68c8224c9e7a60d0c7be10133241887abb5bc4e2fee6d1a364c WatchSource:0}: Error finding container 57262c96f827a68c8224c9e7a60d0c7be10133241887abb5bc4e2fee6d1a364c: Status 404 returned error can't find the container with id 57262c96f827a68c8224c9e7a60d0c7be10133241887abb5bc4e2fee6d1a364c Jul 08 19:19:40.329744 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.329549 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:19:40.329744 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.329691 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:19:40.329997 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:40.329820 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:19:41.019284 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:41.019250 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" event={"ID":"1890d3fc-58bc-446b-b53f-300a7b637bab","Type":"ContainerStarted","Data":"a37cc3f098af843d580a1e06fb70217469f7a0af1e08c1030b5b18f3417e0247"} Jul 08 19:19:41.019692 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:41.019330 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" event={"ID":"1890d3fc-58bc-446b-b53f-300a7b637bab","Type":"ContainerStarted","Data":"57262c96f827a68c8224c9e7a60d0c7be10133241887abb5bc4e2fee6d1a364c"} Jul 08 19:19:41.044604 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:41.044551 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" podStartSLOduration=2.044535314 podStartE2EDuration="2.044535314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:19:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:19:41.043745225 +0000 UTC m=+489.712996916" watchObservedRunningTime="2026-07-08 19:19:41.044535314 +0000 UTC m=+489.713787004" Jul 08 19:19:41.185784 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:41.185753 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:41.191564 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:41.191539 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:42.023085 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:42.023052 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:42.024069 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:42.024053 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:19:42.474757 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:42.474659 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:42.581183 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:42.581139 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:42.646157 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:42.646124 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:43.298609 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:43.298580 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:19:43.368054 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:19:43.368022 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:03.412537 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:03.412498 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:03.517105 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:03.517065 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:03.829972 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:03.829939 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:16.123902 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:16.123861 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:16.678915 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:16.678863 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:22.333270 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.333233 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:22.388774 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.388744 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8d7c58fcd-r6b78"] Jul 08 19:20:22.391931 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.391915 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8d7c58fcd-r6b78" Jul 08 19:20:22.395711 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.395687 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 08 19:20:22.395831 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.395689 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zfzzn\"" Jul 08 19:20:22.396250 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.396228 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 08 19:20:22.407096 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.407069 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8d7c58fcd-r6b78"] Jul 08 19:20:22.458811 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.458773 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:22.544564 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.544529 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:22.557644 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.557614 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd2mk\" (UniqueName: \"kubernetes.io/projected/b3ac27e2-8e22-4c16-b066-c847e7add0e3-kube-api-access-qd2mk\") pod \"maas-controller-8d7c58fcd-r6b78\" (UID: \"b3ac27e2-8e22-4c16-b066-c847e7add0e3\") " pod="opendatahub/maas-controller-8d7c58fcd-r6b78" Jul 08 19:20:22.557761 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.557678 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b3ac27e2-8e22-4c16-b066-c847e7add0e3-webhook-cert\") pod \"maas-controller-8d7c58fcd-r6b78\" (UID: \"b3ac27e2-8e22-4c16-b066-c847e7add0e3\") " pod="opendatahub/maas-controller-8d7c58fcd-r6b78" Jul 08 19:20:22.584932 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.584871 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c9c9cf5d4-v42z9"] Jul 08 19:20:22.587997 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.587984 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" Jul 08 19:20:22.607403 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.607365 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c9c9cf5d4-v42z9"] Jul 08 19:20:22.659061 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.659032 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7gwx\" (UniqueName: \"kubernetes.io/projected/97c9bbd9-c011-4643-be34-975d649ab1e2-kube-api-access-q7gwx\") pod \"maas-controller-c9c9cf5d4-v42z9\" (UID: \"97c9bbd9-c011-4643-be34-975d649ab1e2\") " pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" Jul 08 19:20:22.659210 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.659068 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd2mk\" (UniqueName: \"kubernetes.io/projected/b3ac27e2-8e22-4c16-b066-c847e7add0e3-kube-api-access-qd2mk\") pod \"maas-controller-8d7c58fcd-r6b78\" (UID: \"b3ac27e2-8e22-4c16-b066-c847e7add0e3\") " pod="opendatahub/maas-controller-8d7c58fcd-r6b78" Jul 08 19:20:22.659210 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.659095 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97c9bbd9-c011-4643-be34-975d649ab1e2-webhook-cert\") pod \"maas-controller-c9c9cf5d4-v42z9\" (UID: \"97c9bbd9-c011-4643-be34-975d649ab1e2\") " pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" Jul 08 19:20:22.659210 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.659190 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b3ac27e2-8e22-4c16-b066-c847e7add0e3-webhook-cert\") pod \"maas-controller-8d7c58fcd-r6b78\" (UID: \"b3ac27e2-8e22-4c16-b066-c847e7add0e3\") " pod="opendatahub/maas-controller-8d7c58fcd-r6b78" Jul 08 19:20:22.661658 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.661639 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b3ac27e2-8e22-4c16-b066-c847e7add0e3-webhook-cert\") pod \"maas-controller-8d7c58fcd-r6b78\" (UID: \"b3ac27e2-8e22-4c16-b066-c847e7add0e3\") " pod="opendatahub/maas-controller-8d7c58fcd-r6b78" Jul 08 19:20:22.671882 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.671861 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd2mk\" (UniqueName: \"kubernetes.io/projected/b3ac27e2-8e22-4c16-b066-c847e7add0e3-kube-api-access-qd2mk\") pod \"maas-controller-8d7c58fcd-r6b78\" (UID: \"b3ac27e2-8e22-4c16-b066-c847e7add0e3\") " pod="opendatahub/maas-controller-8d7c58fcd-r6b78" Jul 08 19:20:22.703806 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.703785 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8d7c58fcd-r6b78" Jul 08 19:20:22.760177 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.760147 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7gwx\" (UniqueName: \"kubernetes.io/projected/97c9bbd9-c011-4643-be34-975d649ab1e2-kube-api-access-q7gwx\") pod \"maas-controller-c9c9cf5d4-v42z9\" (UID: \"97c9bbd9-c011-4643-be34-975d649ab1e2\") " pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" Jul 08 19:20:22.760341 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.760220 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97c9bbd9-c011-4643-be34-975d649ab1e2-webhook-cert\") pod \"maas-controller-c9c9cf5d4-v42z9\" (UID: \"97c9bbd9-c011-4643-be34-975d649ab1e2\") " pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" Jul 08 19:20:22.762850 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.762819 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97c9bbd9-c011-4643-be34-975d649ab1e2-webhook-cert\") pod \"maas-controller-c9c9cf5d4-v42z9\" (UID: \"97c9bbd9-c011-4643-be34-975d649ab1e2\") " pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" Jul 08 19:20:22.773651 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.773624 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7gwx\" (UniqueName: \"kubernetes.io/projected/97c9bbd9-c011-4643-be34-975d649ab1e2-kube-api-access-q7gwx\") pod \"maas-controller-c9c9cf5d4-v42z9\" (UID: \"97c9bbd9-c011-4643-be34-975d649ab1e2\") " pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" Jul 08 19:20:22.841278 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.841239 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8d7c58fcd-r6b78"] Jul 08 19:20:22.843432 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:20:22.843402 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ac27e2_8e22_4c16_b066_c847e7add0e3.slice/crio-05d329d3d5634f46b8a3657665e830d01a303891990c74473af6b2df31e14836 WatchSource:0}: Error finding container 05d329d3d5634f46b8a3657665e830d01a303891990c74473af6b2df31e14836: Status 404 returned error can't find the container with id 05d329d3d5634f46b8a3657665e830d01a303891990c74473af6b2df31e14836 Jul 08 19:20:22.897456 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:22.897428 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" Jul 08 19:20:23.028926 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:23.028902 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c9c9cf5d4-v42z9"] Jul 08 19:20:23.030511 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:20:23.030489 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97c9bbd9_c011_4643_be34_975d649ab1e2.slice/crio-00ea7b5519f7261f5b917198117bd5eda272a17f4d05ed3618ea0f943c905115 WatchSource:0}: Error finding container 00ea7b5519f7261f5b917198117bd5eda272a17f4d05ed3618ea0f943c905115: Status 404 returned error can't find the container with id 00ea7b5519f7261f5b917198117bd5eda272a17f4d05ed3618ea0f943c905115 Jul 08 19:20:23.198443 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:23.198342 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" event={"ID":"97c9bbd9-c011-4643-be34-975d649ab1e2","Type":"ContainerStarted","Data":"00ea7b5519f7261f5b917198117bd5eda272a17f4d05ed3618ea0f943c905115"} Jul 08 19:20:23.199429 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:23.199402 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8d7c58fcd-r6b78" event={"ID":"b3ac27e2-8e22-4c16-b066-c847e7add0e3","Type":"ContainerStarted","Data":"05d329d3d5634f46b8a3657665e830d01a303891990c74473af6b2df31e14836"} Jul 08 19:20:25.157045 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:25.156945 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:25.210607 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:25.210573 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" event={"ID":"97c9bbd9-c011-4643-be34-975d649ab1e2","Type":"ContainerStarted","Data":"06ecac86a58438641bd4a5cd2e3796e09ba2d99552bede54c07f240d4643fde2"} Jul 08 19:20:25.210793 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:25.210661 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" Jul 08 19:20:25.212226 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:25.212206 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8d7c58fcd-r6b78" event={"ID":"b3ac27e2-8e22-4c16-b066-c847e7add0e3","Type":"ContainerStarted","Data":"b72c3d050ee7e5edf0bd9e088b7af2f66916adb8d9815be62e3ed5e56d2f09a5"} Jul 08 19:20:25.212338 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:25.212246 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8d7c58fcd-r6b78" Jul 08 19:20:25.239583 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:25.239535 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" podStartSLOduration=1.388854333 podStartE2EDuration="3.239521318s" totalImagesPullingTime="1.850666985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:20:25.234541491 +0000 UTC m=+533.903793181" watchObservedRunningTime="2026-07-08 19:20:25.239521318 +0000 UTC m=+533.908773052" Jul 08 19:20:25.249080 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:25.249049 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:25.272391 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:25.272326 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8d7c58fcd-r6b78" podStartSLOduration=1.244219781 podStartE2EDuration="3.272309306s" totalImagesPullingTime="2.028089525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:20:25.270879071 +0000 UTC m=+533.940130796" watchObservedRunningTime="2026-07-08 19:20:25.272309306 +0000 UTC m=+533.941560996" Jul 08 19:20:36.222488 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:36.222457 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8d7c58fcd-r6b78" Jul 08 19:20:36.222944 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:36.222536 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" Jul 08 19:20:36.288875 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:36.288847 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8d7c58fcd-r6b78"] Jul 08 19:20:36.289096 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:36.289072 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-8d7c58fcd-r6b78" podUID="b3ac27e2-8e22-4c16-b066-c847e7add0e3" containerName="manager" containerID="cri-o://b72c3d050ee7e5edf0bd9e088b7af2f66916adb8d9815be62e3ed5e56d2f09a5" gracePeriod=30 Jul 08 19:20:36.539302 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:36.539279 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8d7c58fcd-r6b78" Jul 08 19:20:36.589081 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:36.589049 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b3ac27e2-8e22-4c16-b066-c847e7add0e3-webhook-cert\") pod \"b3ac27e2-8e22-4c16-b066-c847e7add0e3\" (UID: \"b3ac27e2-8e22-4c16-b066-c847e7add0e3\") " Jul 08 19:20:36.589081 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:36.589086 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd2mk\" (UniqueName: \"kubernetes.io/projected/b3ac27e2-8e22-4c16-b066-c847e7add0e3-kube-api-access-qd2mk\") pod \"b3ac27e2-8e22-4c16-b066-c847e7add0e3\" (UID: \"b3ac27e2-8e22-4c16-b066-c847e7add0e3\") " Jul 08 19:20:36.591218 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:36.591186 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3ac27e2-8e22-4c16-b066-c847e7add0e3-kube-api-access-qd2mk" (OuterVolumeSpecName: "kube-api-access-qd2mk") pod "b3ac27e2-8e22-4c16-b066-c847e7add0e3" (UID: "b3ac27e2-8e22-4c16-b066-c847e7add0e3"). InnerVolumeSpecName "kube-api-access-qd2mk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:20:36.591218 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:36.591196 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3ac27e2-8e22-4c16-b066-c847e7add0e3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b3ac27e2-8e22-4c16-b066-c847e7add0e3" (UID: "b3ac27e2-8e22-4c16-b066-c847e7add0e3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:20:36.690153 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:36.690123 2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b3ac27e2-8e22-4c16-b066-c847e7add0e3-webhook-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:20:36.690153 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:36.690148 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd2mk\" (UniqueName: \"kubernetes.io/projected/b3ac27e2-8e22-4c16-b066-c847e7add0e3-kube-api-access-qd2mk\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:20:37.266024 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:37.265989 2563 generic.go:356] "Generic (PLEG): container finished" podID="b3ac27e2-8e22-4c16-b066-c847e7add0e3" containerID="b72c3d050ee7e5edf0bd9e088b7af2f66916adb8d9815be62e3ed5e56d2f09a5" exitCode=0 Jul 08 19:20:37.266455 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:37.266032 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8d7c58fcd-r6b78" Jul 08 19:20:37.266455 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:37.266070 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8d7c58fcd-r6b78" event={"ID":"b3ac27e2-8e22-4c16-b066-c847e7add0e3","Type":"ContainerDied","Data":"b72c3d050ee7e5edf0bd9e088b7af2f66916adb8d9815be62e3ed5e56d2f09a5"} Jul 08 19:20:37.266455 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:37.266119 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8d7c58fcd-r6b78" event={"ID":"b3ac27e2-8e22-4c16-b066-c847e7add0e3","Type":"ContainerDied","Data":"05d329d3d5634f46b8a3657665e830d01a303891990c74473af6b2df31e14836"} Jul 08 19:20:37.266455 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:37.266139 2563 scope.go:117] "RemoveContainer" containerID="b72c3d050ee7e5edf0bd9e088b7af2f66916adb8d9815be62e3ed5e56d2f09a5" Jul 08 19:20:37.274549 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:37.274530 2563 scope.go:117] "RemoveContainer" containerID="b72c3d050ee7e5edf0bd9e088b7af2f66916adb8d9815be62e3ed5e56d2f09a5" Jul 08 19:20:37.274813 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:20:37.274793 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b72c3d050ee7e5edf0bd9e088b7af2f66916adb8d9815be62e3ed5e56d2f09a5\": container with ID starting with b72c3d050ee7e5edf0bd9e088b7af2f66916adb8d9815be62e3ed5e56d2f09a5 not found: ID does not exist" containerID="b72c3d050ee7e5edf0bd9e088b7af2f66916adb8d9815be62e3ed5e56d2f09a5" Jul 08 19:20:37.274889 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:37.274819 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b72c3d050ee7e5edf0bd9e088b7af2f66916adb8d9815be62e3ed5e56d2f09a5"} err="failed to get container status \"b72c3d050ee7e5edf0bd9e088b7af2f66916adb8d9815be62e3ed5e56d2f09a5\": rpc error: code = NotFound desc = could not find container \"b72c3d050ee7e5edf0bd9e088b7af2f66916adb8d9815be62e3ed5e56d2f09a5\": container with ID starting with b72c3d050ee7e5edf0bd9e088b7af2f66916adb8d9815be62e3ed5e56d2f09a5 not found: ID does not exist" Jul 08 19:20:37.300334 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:37.300310 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8d7c58fcd-r6b78"] Jul 08 19:20:37.309649 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:37.309629 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8d7c58fcd-r6b78"] Jul 08 19:20:37.902891 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:37.902860 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3ac27e2-8e22-4c16-b066-c847e7add0e3" path="/var/lib/kubelet/pods/b3ac27e2-8e22-4c16-b066-c847e7add0e3/volumes" Jul 08 19:20:57.972342 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:57.972278 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:58.538151 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.538113 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:58.577061 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.577027 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-84ff74df46-77c6h"] Jul 08 19:20:58.577487 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.577472 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3ac27e2-8e22-4c16-b066-c847e7add0e3" containerName="manager" Jul 08 19:20:58.577539 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.577491 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3ac27e2-8e22-4c16-b066-c847e7add0e3" containerName="manager" Jul 08 19:20:58.577580 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.577569 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3ac27e2-8e22-4c16-b066-c847e7add0e3" containerName="manager" Jul 08 19:20:58.580664 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.580643 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" Jul 08 19:20:58.587309 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.587276 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 08 19:20:58.587468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.587314 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-58xsg\"" Jul 08 19:20:58.588465 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.588438 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 19:20:58.588619 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.588445 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 19:20:58.590557 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.590537 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84ff74df46-77c6h"] Jul 08 19:20:58.667931 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.667886 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m57nb\" (UniqueName: \"kubernetes.io/projected/e22c3ea9-f46a-4023-a8c0-69d075461840-kube-api-access-m57nb\") pod \"maas-api-84ff74df46-77c6h\" (UID: \"e22c3ea9-f46a-4023-a8c0-69d075461840\") " pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" Jul 08 19:20:58.668122 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.668031 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e22c3ea9-f46a-4023-a8c0-69d075461840-maas-api-tls\") pod \"maas-api-84ff74df46-77c6h\" (UID: \"e22c3ea9-f46a-4023-a8c0-69d075461840\") " pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" Jul 08 19:20:58.668122 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.668059 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e22c3ea9-f46a-4023-a8c0-69d075461840-openshift-service-ca\") pod \"maas-api-84ff74df46-77c6h\" (UID: \"e22c3ea9-f46a-4023-a8c0-69d075461840\") " pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" Jul 08 19:20:58.769497 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.769463 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e22c3ea9-f46a-4023-a8c0-69d075461840-maas-api-tls\") pod \"maas-api-84ff74df46-77c6h\" (UID: \"e22c3ea9-f46a-4023-a8c0-69d075461840\") " pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" Jul 08 19:20:58.769497 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.769500 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e22c3ea9-f46a-4023-a8c0-69d075461840-openshift-service-ca\") pod \"maas-api-84ff74df46-77c6h\" (UID: \"e22c3ea9-f46a-4023-a8c0-69d075461840\") " pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" Jul 08 19:20:58.769737 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.769561 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m57nb\" (UniqueName: \"kubernetes.io/projected/e22c3ea9-f46a-4023-a8c0-69d075461840-kube-api-access-m57nb\") pod \"maas-api-84ff74df46-77c6h\" (UID: \"e22c3ea9-f46a-4023-a8c0-69d075461840\") " pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" Jul 08 19:20:58.769737 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:20:58.769618 2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 08 19:20:58.769737 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:20:58.769698 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e22c3ea9-f46a-4023-a8c0-69d075461840-maas-api-tls podName:e22c3ea9-f46a-4023-a8c0-69d075461840 nodeName:}" failed. No retries permitted until 2026-07-08 19:20:59.269677561 +0000 UTC m=+567.938929246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e22c3ea9-f46a-4023-a8c0-69d075461840-maas-api-tls") pod "maas-api-84ff74df46-77c6h" (UID: "e22c3ea9-f46a-4023-a8c0-69d075461840") : secret "maas-api-serving-cert" not found Jul 08 19:20:58.770131 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.770110 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e22c3ea9-f46a-4023-a8c0-69d075461840-openshift-service-ca\") pod \"maas-api-84ff74df46-77c6h\" (UID: \"e22c3ea9-f46a-4023-a8c0-69d075461840\") " pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" Jul 08 19:20:58.779126 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:58.779104 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m57nb\" (UniqueName: \"kubernetes.io/projected/e22c3ea9-f46a-4023-a8c0-69d075461840-kube-api-access-m57nb\") pod \"maas-api-84ff74df46-77c6h\" (UID: \"e22c3ea9-f46a-4023-a8c0-69d075461840\") " pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" Jul 08 19:20:59.010017 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.009979 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:59.217429 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.217395 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-hf2nf"] Jul 08 19:20:59.223226 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.223205 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-hf2nf" Jul 08 19:20:59.227149 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.227127 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 08 19:20:59.227242 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.227156 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9sp4f\"" Jul 08 19:20:59.236502 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.236482 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-hf2nf"] Jul 08 19:20:59.252891 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.252867 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-9nskb"] Jul 08 19:20:59.257020 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.257005 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-9nskb" Jul 08 19:20:59.267176 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.267157 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-9nskb"] Jul 08 19:20:59.273928 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.273900 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e22c3ea9-f46a-4023-a8c0-69d075461840-maas-api-tls\") pod \"maas-api-84ff74df46-77c6h\" (UID: \"e22c3ea9-f46a-4023-a8c0-69d075461840\") " pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" Jul 08 19:20:59.274016 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.273942 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcfcc\" (UniqueName: \"kubernetes.io/projected/68c23503-4cdc-44d9-b1f1-60dbadd5793b-kube-api-access-jcfcc\") pod \"payload-pre-processing-7964457696-hf2nf\" (UID: \"68c23503-4cdc-44d9-b1f1-60dbadd5793b\") " pod="openshift-ingress/payload-pre-processing-7964457696-hf2nf" Jul 08 19:20:59.274016 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.274004 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68c23503-4cdc-44d9-b1f1-60dbadd5793b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-hf2nf\" (UID: \"68c23503-4cdc-44d9-b1f1-60dbadd5793b\") " pod="openshift-ingress/payload-pre-processing-7964457696-hf2nf" Jul 08 19:20:59.274294 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.274266 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:59.276428 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.276399 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e22c3ea9-f46a-4023-a8c0-69d075461840-maas-api-tls\") pod \"maas-api-84ff74df46-77c6h\" (UID: \"e22c3ea9-f46a-4023-a8c0-69d075461840\") " pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" Jul 08 19:20:59.352579 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.352539 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:20:59.374702 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.374675 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcfcc\" (UniqueName: \"kubernetes.io/projected/68c23503-4cdc-44d9-b1f1-60dbadd5793b-kube-api-access-jcfcc\") pod \"payload-pre-processing-7964457696-hf2nf\" (UID: \"68c23503-4cdc-44d9-b1f1-60dbadd5793b\") " pod="openshift-ingress/payload-pre-processing-7964457696-hf2nf" Jul 08 19:20:59.374850 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.374713 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j6jj\" (UniqueName: \"kubernetes.io/projected/1d6395bc-72fc-4973-a0ad-53909b703a42-kube-api-access-2j6jj\") pod \"payload-processing-58f86c7798-9nskb\" (UID: \"1d6395bc-72fc-4973-a0ad-53909b703a42\") " pod="openshift-ingress/payload-processing-58f86c7798-9nskb" Jul 08 19:20:59.374850 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.374767 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68c23503-4cdc-44d9-b1f1-60dbadd5793b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-hf2nf\" (UID: \"68c23503-4cdc-44d9-b1f1-60dbadd5793b\") " pod="openshift-ingress/payload-pre-processing-7964457696-hf2nf" Jul 08 19:20:59.374850 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.374784 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d6395bc-72fc-4973-a0ad-53909b703a42-plugins-config-volume\") pod \"payload-processing-58f86c7798-9nskb\" (UID: \"1d6395bc-72fc-4973-a0ad-53909b703a42\") " pod="openshift-ingress/payload-processing-58f86c7798-9nskb" Jul 08 19:20:59.375427 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.375409 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68c23503-4cdc-44d9-b1f1-60dbadd5793b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-hf2nf\" (UID: \"68c23503-4cdc-44d9-b1f1-60dbadd5793b\") " pod="openshift-ingress/payload-pre-processing-7964457696-hf2nf" Jul 08 19:20:59.386818 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.386793 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcfcc\" (UniqueName: \"kubernetes.io/projected/68c23503-4cdc-44d9-b1f1-60dbadd5793b-kube-api-access-jcfcc\") pod \"payload-pre-processing-7964457696-hf2nf\" (UID: \"68c23503-4cdc-44d9-b1f1-60dbadd5793b\") " pod="openshift-ingress/payload-pre-processing-7964457696-hf2nf" Jul 08 19:20:59.475648 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.475603 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j6jj\" (UniqueName: \"kubernetes.io/projected/1d6395bc-72fc-4973-a0ad-53909b703a42-kube-api-access-2j6jj\") pod \"payload-processing-58f86c7798-9nskb\" (UID: \"1d6395bc-72fc-4973-a0ad-53909b703a42\") " pod="openshift-ingress/payload-processing-58f86c7798-9nskb" Jul 08 19:20:59.475821 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.475702 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d6395bc-72fc-4973-a0ad-53909b703a42-plugins-config-volume\") pod \"payload-processing-58f86c7798-9nskb\" (UID: \"1d6395bc-72fc-4973-a0ad-53909b703a42\") " pod="openshift-ingress/payload-processing-58f86c7798-9nskb" Jul 08 19:20:59.476674 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.476650 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d6395bc-72fc-4973-a0ad-53909b703a42-plugins-config-volume\") pod \"payload-processing-58f86c7798-9nskb\" (UID: \"1d6395bc-72fc-4973-a0ad-53909b703a42\") " pod="openshift-ingress/payload-processing-58f86c7798-9nskb" Jul 08 19:20:59.485087 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.485062 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j6jj\" (UniqueName: \"kubernetes.io/projected/1d6395bc-72fc-4973-a0ad-53909b703a42-kube-api-access-2j6jj\") pod \"payload-processing-58f86c7798-9nskb\" (UID: \"1d6395bc-72fc-4973-a0ad-53909b703a42\") " pod="openshift-ingress/payload-processing-58f86c7798-9nskb" Jul 08 19:20:59.497753 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.497731 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" Jul 08 19:20:59.533211 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.533130 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-hf2nf" Jul 08 19:20:59.567400 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.567356 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-9nskb" Jul 08 19:20:59.646531 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.646477 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84ff74df46-77c6h"] Jul 08 19:20:59.649522 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:20:59.649443 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode22c3ea9_f46a_4023_a8c0_69d075461840.slice/crio-412afe2029498f0cb0ffcf4522749d6686c621babbd71dae03ea1bc02d4318c8 WatchSource:0}: Error finding container 412afe2029498f0cb0ffcf4522749d6686c621babbd71dae03ea1bc02d4318c8: Status 404 returned error can't find the container with id 412afe2029498f0cb0ffcf4522749d6686c621babbd71dae03ea1bc02d4318c8 Jul 08 19:20:59.692297 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.692267 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-hf2nf"] Jul 08 19:20:59.697036 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:20:59.696908 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68c23503_4cdc_44d9_b1f1_60dbadd5793b.slice/crio-9ce05805c183c1a85c6090f9731142339e34c9016a38464661c4770d675ff3a3 WatchSource:0}: Error finding container 9ce05805c183c1a85c6090f9731142339e34c9016a38464661c4770d675ff3a3: Status 404 returned error can't find the container with id 9ce05805c183c1a85c6090f9731142339e34c9016a38464661c4770d675ff3a3 Jul 08 19:20:59.727891 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:20:59.727854 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-9nskb"] Jul 08 19:20:59.731513 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:20:59.731483 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d6395bc_72fc_4973_a0ad_53909b703a42.slice/crio-3badd20b9b19901ba12aad1baf0e2d2636be7feb62d81354ef2ef2d559e06319 WatchSource:0}: Error finding container 3badd20b9b19901ba12aad1baf0e2d2636be7feb62d81354ef2ef2d559e06319: Status 404 returned error can't find the container with id 3badd20b9b19901ba12aad1baf0e2d2636be7feb62d81354ef2ef2d559e06319 Jul 08 19:21:00.366143 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:00.366109 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-hf2nf" event={"ID":"68c23503-4cdc-44d9-b1f1-60dbadd5793b","Type":"ContainerStarted","Data":"9ce05805c183c1a85c6090f9731142339e34c9016a38464661c4770d675ff3a3"} Jul 08 19:21:00.367298 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:00.367273 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" event={"ID":"e22c3ea9-f46a-4023-a8c0-69d075461840","Type":"ContainerStarted","Data":"412afe2029498f0cb0ffcf4522749d6686c621babbd71dae03ea1bc02d4318c8"} Jul 08 19:21:00.368393 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:00.368360 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-9nskb" event={"ID":"1d6395bc-72fc-4973-a0ad-53909b703a42","Type":"ContainerStarted","Data":"3badd20b9b19901ba12aad1baf0e2d2636be7feb62d81354ef2ef2d559e06319"} Jul 08 19:21:02.381832 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:02.381747 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" event={"ID":"e22c3ea9-f46a-4023-a8c0-69d075461840","Type":"ContainerStarted","Data":"8ba59325a28d9b5e6d0f07cce8d02b6518a47f5e38811b2d8cc9039fc2cb9f0f"} Jul 08 19:21:02.382220 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:02.381858 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" Jul 08 19:21:02.383559 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:02.383532 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-9nskb" event={"ID":"1d6395bc-72fc-4973-a0ad-53909b703a42","Type":"ContainerStarted","Data":"dbb667cb901a76270505ec3de285006b12cd5a2ede9c1aaf0de8e8c19972ec95"} Jul 08 19:21:02.383677 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:02.383615 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-9nskb" Jul 08 19:21:02.385187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:02.385161 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-hf2nf" event={"ID":"68c23503-4cdc-44d9-b1f1-60dbadd5793b","Type":"ContainerStarted","Data":"e89313451fd4f0b99bbf2c4b416c1ae4c3c1dbed3b8abdbff671762bf32f7718"} Jul 08 19:21:02.385298 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:02.385284 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-hf2nf" Jul 08 19:21:02.419591 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:02.419540 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" podStartSLOduration=1.978639281 podStartE2EDuration="4.419520415s" totalImagesPullingTime="2.440881134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:21:02.417753596 +0000 UTC m=+571.087005299" watchObservedRunningTime="2026-07-08 19:21:02.419520415 +0000 UTC m=+571.088772108" Jul 08 19:21:02.485093 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:02.485034 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-hf2nf" podStartSLOduration=1.090750815 podStartE2EDuration="3.485010163s" totalImagesPullingTime="2.394259348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:20:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:21:02.463564792 +0000 UTC m=+571.132816483" watchObservedRunningTime="2026-07-08 19:21:02.485010163 +0000 UTC m=+571.154261855" Jul 08 19:21:02.485365 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:02.485340 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-9nskb" podStartSLOduration=1.127186866 podStartE2EDuration="3.485332149s" totalImagesPullingTime="2.358145283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:20:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:21:02.484035605 +0000 UTC m=+571.153287296" watchObservedRunningTime="2026-07-08 19:21:02.485332149 +0000 UTC m=+571.154583842" Jul 08 19:21:04.635187 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:04.635123 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:21:04.735656 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:04.735609 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:21:07.409365 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:07.409333 2563 generic.go:356] "Generic (PLEG): container finished" podID="1890d3fc-58bc-446b-b53f-300a7b637bab" containerID="a37cc3f098af843d580a1e06fb70217469f7a0af1e08c1030b5b18f3417e0247" exitCode=0 Jul 08 19:21:07.409787 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:07.409411 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" event={"ID":"1890d3fc-58bc-446b-b53f-300a7b637bab","Type":"ContainerDied","Data":"a37cc3f098af843d580a1e06fb70217469f7a0af1e08c1030b5b18f3417e0247"} Jul 08 19:21:07.409828 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:07.409791 2563 scope.go:117] "RemoveContainer" containerID="a37cc3f098af843d580a1e06fb70217469f7a0af1e08c1030b5b18f3417e0247" Jul 08 19:21:07.410838 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:07.410813 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:21:07.410900 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:07.410878 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:21:07.410938 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:07.410908 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:21:08.396445 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:08.396414 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-84ff74df46-77c6h" Jul 08 19:21:08.416608 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:08.416576 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" event={"ID":"1890d3fc-58bc-446b-b53f-300a7b637bab","Type":"ContainerStarted","Data":"8be3667f3283b3ac7627a9fb83762bba4145e70da4b4bb885687950fd4a4c0f1"} Jul 08 19:21:09.185790 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:09.185749 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:21:09.186982 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:09.186953 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xg7ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 08 19:21:09.187117 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:09.187010 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" podUID="1890d3fc-58bc-446b-b53f-300a7b637bab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 08 19:21:09.417398 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:09.417333 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:21:09.515103 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:09.515072 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:21:10.185250 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:10.185214 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xg7ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 08 19:21:10.185450 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:10.185267 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" podUID="1890d3fc-58bc-446b-b53f-300a7b637bab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 08 19:21:11.185999 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:11.185963 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xg7ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 08 19:21:11.186471 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:11.186035 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" podUID="1890d3fc-58bc-446b-b53f-300a7b637bab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 08 19:21:12.023894 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:12.023859 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:21:12.186277 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:12.186237 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xg7ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 08 19:21:12.186692 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:12.186298 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" podUID="1890d3fc-58bc-446b-b53f-300a7b637bab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 08 19:21:13.189940 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:13.189908 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:21:13.190878 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:13.190857 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xg7ds" Jul 08 19:21:13.391959 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:13.391931 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-9nskb" Jul 08 19:21:13.392155 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:13.392094 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-hf2nf" Jul 08 19:21:17.490165 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:17.490132 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:21:22.104393 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:22.104342 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:21:22.166790 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:22.166747 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:21:28.319461 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.319429 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6"] Jul 08 19:21:28.322977 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.322960 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.325791 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.325769 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 19:21:28.325934 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.325772 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 19:21:28.327016 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.326997 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-h65ml\"" Jul 08 19:21:28.327155 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.327139 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 08 19:21:28.335426 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.335398 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6"] Jul 08 19:21:28.345033 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.345009 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:21:28.426334 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.426303 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be6655e0-b40e-4247-b9c0-f2e065b1b846-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.426526 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.426364 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzrvv\" (UniqueName: \"kubernetes.io/projected/be6655e0-b40e-4247-b9c0-f2e065b1b846-kube-api-access-tzrvv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.426526 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.426474 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be6655e0-b40e-4247-b9c0-f2e065b1b846-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.426526 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.426512 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be6655e0-b40e-4247-b9c0-f2e065b1b846-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.426640 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.426530 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be6655e0-b40e-4247-b9c0-f2e065b1b846-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.426640 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.426569 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be6655e0-b40e-4247-b9c0-f2e065b1b846-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.527500 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.527460 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be6655e0-b40e-4247-b9c0-f2e065b1b846-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.527500 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.527500 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be6655e0-b40e-4247-b9c0-f2e065b1b846-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.527727 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.527521 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be6655e0-b40e-4247-b9c0-f2e065b1b846-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.527727 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.527642 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be6655e0-b40e-4247-b9c0-f2e065b1b846-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.527846 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.527739 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzrvv\" (UniqueName: \"kubernetes.io/projected/be6655e0-b40e-4247-b9c0-f2e065b1b846-kube-api-access-tzrvv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.527846 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.527800 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be6655e0-b40e-4247-b9c0-f2e065b1b846-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.527958 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.527935 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be6655e0-b40e-4247-b9c0-f2e065b1b846-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.528013 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.527954 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be6655e0-b40e-4247-b9c0-f2e065b1b846-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.528137 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.528118 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be6655e0-b40e-4247-b9c0-f2e065b1b846-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.529790 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.529774 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be6655e0-b40e-4247-b9c0-f2e065b1b846-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.530084 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.530068 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be6655e0-b40e-4247-b9c0-f2e065b1b846-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.536347 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.536321 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzrvv\" (UniqueName: \"kubernetes.io/projected/be6655e0-b40e-4247-b9c0-f2e065b1b846-kube-api-access-tzrvv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rmnc6\" (UID: \"be6655e0-b40e-4247-b9c0-f2e065b1b846\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.635207 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.635120 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:28.766165 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:28.766134 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6"] Jul 08 19:21:28.768300 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:21:28.768274 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe6655e0_b40e_4247_b9c0_f2e065b1b846.slice/crio-1ffe67f13ab513e26eb6cadacaa3c759006d3b064a8bfd711ace0e7d7a0b6d31 WatchSource:0}: Error finding container 1ffe67f13ab513e26eb6cadacaa3c759006d3b064a8bfd711ace0e7d7a0b6d31: Status 404 returned error can't find the container with id 1ffe67f13ab513e26eb6cadacaa3c759006d3b064a8bfd711ace0e7d7a0b6d31 Jul 08 19:21:29.519479 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:29.519419 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" event={"ID":"be6655e0-b40e-4247-b9c0-f2e065b1b846","Type":"ContainerStarted","Data":"1ffe67f13ab513e26eb6cadacaa3c759006d3b064a8bfd711ace0e7d7a0b6d31"} Jul 08 19:21:30.525357 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:30.525320 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" event={"ID":"be6655e0-b40e-4247-b9c0-f2e065b1b846","Type":"ContainerStarted","Data":"91a18ecdd4aee6cf5bc6f1a4a65b16fed64c55b05bd01dbc980e01d9ef7c5961"} Jul 08 19:21:30.549189 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:30.549134 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" podStartSLOduration=1.547860177 podStartE2EDuration="2.549116428s" totalImagesPullingTime="1.001256251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:21:30.547791604 +0000 UTC m=+599.217043302" watchObservedRunningTime="2026-07-08 19:21:30.549116428 +0000 UTC m=+599.218368117" Jul 08 19:21:31.757299 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:31.757269 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:21:31.758192 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:31.758174 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:21:37.617801 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:37.617760 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:21:38.636249 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:38.636217 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:38.636651 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:38.636277 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:38.648577 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:38.648556 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:39.576429 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:39.576402 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rmnc6" Jul 08 19:21:43.712617 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:43.712580 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:21:52.733013 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:21:52.732967 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:22:14.140542 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:22:14.140503 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:22:18.922859 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:22:18.922816 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:22:46.801676 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:22:46.801640 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:23:26.106321 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:23:26.106230 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:23:26.212404 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:23:26.212354 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:23:27.364445 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:23:27.364408 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:23:27.464613 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:23:27.464579 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:24:23.173636 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:23.173597 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:24:38.325832 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:38.325783 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:24:38.352498 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:38.352465 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c9c9cf5d4-v42z9"] Jul 08 19:24:38.352692 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:38.352648 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" podUID="97c9bbd9-c011-4643-be34-975d649ab1e2" containerName="manager" containerID="cri-o://06ecac86a58438641bd4a5cd2e3796e09ba2d99552bede54c07f240d4643fde2" gracePeriod=30 Jul 08 19:24:38.602579 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:38.602552 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" Jul 08 19:24:38.683799 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:38.683769 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7gwx\" (UniqueName: \"kubernetes.io/projected/97c9bbd9-c011-4643-be34-975d649ab1e2-kube-api-access-q7gwx\") pod \"97c9bbd9-c011-4643-be34-975d649ab1e2\" (UID: \"97c9bbd9-c011-4643-be34-975d649ab1e2\") " Jul 08 19:24:38.683974 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:38.683854 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97c9bbd9-c011-4643-be34-975d649ab1e2-webhook-cert\") pod \"97c9bbd9-c011-4643-be34-975d649ab1e2\" (UID: \"97c9bbd9-c011-4643-be34-975d649ab1e2\") " Jul 08 19:24:38.685893 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:38.685869 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97c9bbd9-c011-4643-be34-975d649ab1e2-kube-api-access-q7gwx" (OuterVolumeSpecName: "kube-api-access-q7gwx") pod "97c9bbd9-c011-4643-be34-975d649ab1e2" (UID: "97c9bbd9-c011-4643-be34-975d649ab1e2"). InnerVolumeSpecName "kube-api-access-q7gwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:24:38.685992 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:38.685868 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97c9bbd9-c011-4643-be34-975d649ab1e2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "97c9bbd9-c011-4643-be34-975d649ab1e2" (UID: "97c9bbd9-c011-4643-be34-975d649ab1e2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:24:38.784738 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:38.784700 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7gwx\" (UniqueName: \"kubernetes.io/projected/97c9bbd9-c011-4643-be34-975d649ab1e2-kube-api-access-q7gwx\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:24:38.784738 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:38.784730 2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97c9bbd9-c011-4643-be34-975d649ab1e2-webhook-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:24:39.363316 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:39.363285 2563 generic.go:356] "Generic (PLEG): container finished" podID="97c9bbd9-c011-4643-be34-975d649ab1e2" containerID="06ecac86a58438641bd4a5cd2e3796e09ba2d99552bede54c07f240d4643fde2" exitCode=0 Jul 08 19:24:39.363819 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:39.363329 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" Jul 08 19:24:39.363819 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:39.363332 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" event={"ID":"97c9bbd9-c011-4643-be34-975d649ab1e2","Type":"ContainerDied","Data":"06ecac86a58438641bd4a5cd2e3796e09ba2d99552bede54c07f240d4643fde2"} Jul 08 19:24:39.363819 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:39.363482 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c9c9cf5d4-v42z9" event={"ID":"97c9bbd9-c011-4643-be34-975d649ab1e2","Type":"ContainerDied","Data":"00ea7b5519f7261f5b917198117bd5eda272a17f4d05ed3618ea0f943c905115"} Jul 08 19:24:39.363819 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:39.363525 2563 scope.go:117] "RemoveContainer" containerID="06ecac86a58438641bd4a5cd2e3796e09ba2d99552bede54c07f240d4643fde2" Jul 08 19:24:39.372831 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:39.372807 2563 scope.go:117] "RemoveContainer" containerID="06ecac86a58438641bd4a5cd2e3796e09ba2d99552bede54c07f240d4643fde2" Jul 08 19:24:39.373090 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:24:39.373072 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06ecac86a58438641bd4a5cd2e3796e09ba2d99552bede54c07f240d4643fde2\": container with ID starting with 06ecac86a58438641bd4a5cd2e3796e09ba2d99552bede54c07f240d4643fde2 not found: ID does not exist" containerID="06ecac86a58438641bd4a5cd2e3796e09ba2d99552bede54c07f240d4643fde2" Jul 08 19:24:39.373142 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:39.373099 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06ecac86a58438641bd4a5cd2e3796e09ba2d99552bede54c07f240d4643fde2"} err="failed to get container status \"06ecac86a58438641bd4a5cd2e3796e09ba2d99552bede54c07f240d4643fde2\": rpc error: code = NotFound desc = could not find container \"06ecac86a58438641bd4a5cd2e3796e09ba2d99552bede54c07f240d4643fde2\": container with ID starting with 06ecac86a58438641bd4a5cd2e3796e09ba2d99552bede54c07f240d4643fde2 not found: ID does not exist" Jul 08 19:24:39.387034 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:39.387011 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c9c9cf5d4-v42z9"] Jul 08 19:24:39.391564 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:39.391545 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-c9c9cf5d4-v42z9"] Jul 08 19:24:39.903489 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:39.903455 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97c9bbd9-c011-4643-be34-975d649ab1e2" path="/var/lib/kubelet/pods/97c9bbd9-c011-4643-be34-975d649ab1e2/volumes" Jul 08 19:24:40.076173 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.076125 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:24:40.192979 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.192883 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c9c9cf5d4-x8fvq"] Jul 08 19:24:40.193214 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.193201 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97c9bbd9-c011-4643-be34-975d649ab1e2" containerName="manager" Jul 08 19:24:40.193261 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.193216 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="97c9bbd9-c011-4643-be34-975d649ab1e2" containerName="manager" Jul 08 19:24:40.193295 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.193283 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="97c9bbd9-c011-4643-be34-975d649ab1e2" containerName="manager" Jul 08 19:24:40.197343 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.197327 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c9c9cf5d4-x8fvq" Jul 08 19:24:40.200683 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.200664 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 08 19:24:40.201942 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.201923 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zfzzn\"" Jul 08 19:24:40.202029 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.201949 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 08 19:24:40.207546 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.207518 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c9c9cf5d4-x8fvq"] Jul 08 19:24:40.295840 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.295809 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txsld\" (UniqueName: \"kubernetes.io/projected/ad50df7b-244c-4f7c-9d24-94e324fa5b35-kube-api-access-txsld\") pod \"maas-controller-c9c9cf5d4-x8fvq\" (UID: \"ad50df7b-244c-4f7c-9d24-94e324fa5b35\") " pod="opendatahub/maas-controller-c9c9cf5d4-x8fvq" Jul 08 19:24:40.296005 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.295860 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad50df7b-244c-4f7c-9d24-94e324fa5b35-webhook-cert\") pod \"maas-controller-c9c9cf5d4-x8fvq\" (UID: \"ad50df7b-244c-4f7c-9d24-94e324fa5b35\") " pod="opendatahub/maas-controller-c9c9cf5d4-x8fvq" Jul 08 19:24:40.396724 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.396691 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad50df7b-244c-4f7c-9d24-94e324fa5b35-webhook-cert\") pod \"maas-controller-c9c9cf5d4-x8fvq\" (UID: \"ad50df7b-244c-4f7c-9d24-94e324fa5b35\") " pod="opendatahub/maas-controller-c9c9cf5d4-x8fvq" Jul 08 19:24:40.397211 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.396781 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txsld\" (UniqueName: \"kubernetes.io/projected/ad50df7b-244c-4f7c-9d24-94e324fa5b35-kube-api-access-txsld\") pod \"maas-controller-c9c9cf5d4-x8fvq\" (UID: \"ad50df7b-244c-4f7c-9d24-94e324fa5b35\") " pod="opendatahub/maas-controller-c9c9cf5d4-x8fvq" Jul 08 19:24:40.399214 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.399190 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad50df7b-244c-4f7c-9d24-94e324fa5b35-webhook-cert\") pod \"maas-controller-c9c9cf5d4-x8fvq\" (UID: \"ad50df7b-244c-4f7c-9d24-94e324fa5b35\") " pod="opendatahub/maas-controller-c9c9cf5d4-x8fvq" Jul 08 19:24:40.405878 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.405857 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txsld\" (UniqueName: \"kubernetes.io/projected/ad50df7b-244c-4f7c-9d24-94e324fa5b35-kube-api-access-txsld\") pod \"maas-controller-c9c9cf5d4-x8fvq\" (UID: \"ad50df7b-244c-4f7c-9d24-94e324fa5b35\") " pod="opendatahub/maas-controller-c9c9cf5d4-x8fvq" Jul 08 19:24:40.508698 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.508673 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c9c9cf5d4-x8fvq" Jul 08 19:24:40.640449 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.640424 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c9c9cf5d4-x8fvq"] Jul 08 19:24:40.643015 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:24:40.642982 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad50df7b_244c_4f7c_9d24_94e324fa5b35.slice/crio-4230d7201e9241ea392aca0571929dda61f7333f6b8f76c2eeb974ab8ce96e6b WatchSource:0}: Error finding container 4230d7201e9241ea392aca0571929dda61f7333f6b8f76c2eeb974ab8ce96e6b: Status 404 returned error can't find the container with id 4230d7201e9241ea392aca0571929dda61f7333f6b8f76c2eeb974ab8ce96e6b Jul 08 19:24:40.644362 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:40.644344 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:24:41.375767 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:41.375671 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c9c9cf5d4-x8fvq" event={"ID":"ad50df7b-244c-4f7c-9d24-94e324fa5b35","Type":"ContainerStarted","Data":"873d5493d9a32e3b628bc613df0de1c6b39185989eb0bf39dd967b38dd8e1671"} Jul 08 19:24:41.375767 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:41.375731 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c9c9cf5d4-x8fvq" Jul 08 19:24:41.375767 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:41.375747 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c9c9cf5d4-x8fvq" event={"ID":"ad50df7b-244c-4f7c-9d24-94e324fa5b35","Type":"ContainerStarted","Data":"4230d7201e9241ea392aca0571929dda61f7333f6b8f76c2eeb974ab8ce96e6b"} Jul 08 19:24:41.395746 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:41.395699 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c9c9cf5d4-x8fvq" podStartSLOduration=0.955240252 podStartE2EDuration="1.395685212s" totalImagesPullingTime="440.44496ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:24:41.394711343 +0000 UTC m=+790.063963031" watchObservedRunningTime="2026-07-08 19:24:41.395685212 +0000 UTC m=+790.064936902" Jul 08 19:24:52.387006 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:24:52.386977 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c9c9cf5d4-x8fvq" Jul 08 19:25:07.395081 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:25:07.394983 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:25:07.495029 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:25:07.494986 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:25:07.788130 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:25:07.788092 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:25:21.879807 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:25:21.879760 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:25:25.770757 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:25:25.770722 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:25:25.866337 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:25:25.866303 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:25:26.002342 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:25:26.002303 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:25:31.913871 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:25:31.913837 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:25:32.002891 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:25:32.002846 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:25:32.150069 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:25:32.149970 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:25:32.304956 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:25:32.304915 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:25:38.221217 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:25:38.221180 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:25:38.326074 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:25:38.326035 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:26:31.787351 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:26:31.784067 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:26:31.790280 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:26:31.790255 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:27:02.679074 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:27:02.679034 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:27:04.084665 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:27:04.084626 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:28:31.145893 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:28:31.145859 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:28:31.241265 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:28:31.241230 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:31:31.816828 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:31:31.816774 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:31:31.821174 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:31:31.821148 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:31:38.155951 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:31:38.155918 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:32:47.135425 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:32:47.135365 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:34:13.686177 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:34:13.686085 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:36:28.901752 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:36:28.901711 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:36:31.842456 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:36:31.842321 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:36:31.848942 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:36:31.848914 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:36:39.370532 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:36:39.370496 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:36:46.740530 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:36:46.740492 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:36:51.863095 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:36:51.863058 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:39:39.481361 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:39:39.481328 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:39:51.872244 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:39:51.872205 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:39:51.972285 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:39:51.972255 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:39:54.822585 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:39:54.822509 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:41:31.871910 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:41:31.871802 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:41:31.877877 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:41:31.876877 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:43:04.058991 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:04.058908 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:04.145412 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:04.145365 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:04.487416 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:04.487307 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:10.204458 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:10.204419 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:10.305442 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:10.305350 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:10.415641 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:10.415605 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:10.516207 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:10.516166 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:11.509031 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:11.508990 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:11.609650 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:11.609617 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:15.071286 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:15.071243 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:16.688552 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:16.688506 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:16.858788 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:16.858750 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:16.968936 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:16.968842 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:18.536763 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:18.536728 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:22.911416 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:22.911359 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:23.035277 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:23.035236 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:23.181147 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:23.181058 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:23.245493 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:23.245449 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:24.313929 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:24.313890 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:24.417502 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:24.417464 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:26.453180 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:26.453148 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:29.429185 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:29.429150 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:30.438918 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:30.438887 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:30.539779 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:30.539738 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:33.444138 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:33.444096 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:36.240952 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.240905 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:36.344558 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.344515 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:36.421264 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.421230 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf"] Jul 08 19:43:36.424847 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.424819 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.427712 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.427688 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-e2758a2d-openshift-default-dockercfg-8cgfm\"" Jul 08 19:43:36.436941 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.436907 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf"] Jul 08 19:43:36.466009 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.465981 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:36.486156 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.486124 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-podinfo\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.486290 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.486185 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-token\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.486290 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.486215 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istiod-ca-cert\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.486290 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.486245 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkjtc\" (UniqueName: \"kubernetes.io/projected/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-kube-api-access-bkjtc\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.486420 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.486330 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-data\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.486420 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.486403 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-workload-socket\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.486493 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.486436 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-envoy\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.486493 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.486454 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-credential-socket\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.486493 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.486471 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-workload-certs\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.565990 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.565942 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:36.587087 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.587059 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-workload-socket\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.587212 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.587095 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-envoy\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.587212 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.587116 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-credential-socket\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.587212 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.587132 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-workload-certs\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.587212 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.587177 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-podinfo\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.587447 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.587232 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-token\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.587447 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.587252 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istiod-ca-cert\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.587447 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.587281 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkjtc\" (UniqueName: \"kubernetes.io/projected/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-kube-api-access-bkjtc\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.587601 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.587541 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-workload-socket\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.587601 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.587540 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-data\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.587601 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.587590 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-credential-socket\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.587807 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.587779 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-workload-certs\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.587935 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.587913 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-data\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.588147 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.588129 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istiod-ca-cert\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.589511 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.589492 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-envoy\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.589663 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.589647 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-podinfo\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.595424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.595402 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-token\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.595656 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.595638 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkjtc\" (UniqueName: \"kubernetes.io/projected/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-kube-api-access-bkjtc\") pod \"e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.736940 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.736904 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:36.879885 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.879859 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf"] Jul 08 19:43:36.881911 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:43:36.881883 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod700f081f_d65f_41cd_b8ff_8ad7d77ac6ae.slice/crio-a6c7b8b1150791c44ef3026cd195de73507556235b579b256ec1f44f61c256cd WatchSource:0}: Error finding container a6c7b8b1150791c44ef3026cd195de73507556235b579b256ec1f44f61c256cd: Status 404 returned error can't find the container with id a6c7b8b1150791c44ef3026cd195de73507556235b579b256ec1f44f61c256cd Jul 08 19:43:36.884018 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.883975 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:43:36.884106 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.884051 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:43:36.884106 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:36.884083 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:43:37.359789 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:37.359751 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" event={"ID":"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae","Type":"ContainerStarted","Data":"b227abdcf2e60efde91318bda1efe8cff5f16619092abb1f4317b3a97ca77173"} Jul 08 19:43:37.360190 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:37.359805 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" event={"ID":"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae","Type":"ContainerStarted","Data":"a6c7b8b1150791c44ef3026cd195de73507556235b579b256ec1f44f61c256cd"} Jul 08 19:43:37.380892 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:37.380847 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" podStartSLOduration=1.380832631 podStartE2EDuration="1.380832631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:43:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:43:37.379190916 +0000 UTC m=+1926.048442601" watchObservedRunningTime="2026-07-08 19:43:37.380832631 +0000 UTC m=+1926.050084321" Jul 08 19:43:37.571030 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:37.570992 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:37.674528 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:37.674438 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:37.737459 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:37.737421 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:37.742605 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:37.742581 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:38.088723 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.088684 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm"] Jul 08 19:43:38.092266 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.092245 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" Jul 08 19:43:38.094860 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.094835 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-e2758a2d-serving-cert\"" Jul 08 19:43:38.102952 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.102927 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm"] Jul 08 19:43:38.126231 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.126200 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:38.202640 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.202607 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ee4c810-d74a-486f-be94-90f84b5b8ae4-maas-api-tls\") pod \"maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm\" (UID: \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" Jul 08 19:43:38.202822 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.202663 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mntqk\" (UniqueName: \"kubernetes.io/projected/8ee4c810-d74a-486f-be94-90f84b5b8ae4-kube-api-access-mntqk\") pod \"maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm\" (UID: \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" Jul 08 19:43:38.202822 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.202726 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ee4c810-d74a-486f-be94-90f84b5b8ae4-openshift-service-ca\") pod \"maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm\" (UID: \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" Jul 08 19:43:38.246096 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.246062 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:38.304200 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.304162 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ee4c810-d74a-486f-be94-90f84b5b8ae4-maas-api-tls\") pod \"maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm\" (UID: \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" Jul 08 19:43:38.304358 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.304241 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mntqk\" (UniqueName: \"kubernetes.io/projected/8ee4c810-d74a-486f-be94-90f84b5b8ae4-kube-api-access-mntqk\") pod \"maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm\" (UID: \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" Jul 08 19:43:38.304472 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.304420 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ee4c810-d74a-486f-be94-90f84b5b8ae4-openshift-service-ca\") pod \"maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm\" (UID: \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" Jul 08 19:43:38.305069 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.305034 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ee4c810-d74a-486f-be94-90f84b5b8ae4-openshift-service-ca\") pod \"maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm\" (UID: \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" Jul 08 19:43:38.306771 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.306751 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ee4c810-d74a-486f-be94-90f84b5b8ae4-maas-api-tls\") pod \"maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm\" (UID: \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" Jul 08 19:43:38.317305 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.317282 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mntqk\" (UniqueName: \"kubernetes.io/projected/8ee4c810-d74a-486f-be94-90f84b5b8ae4-kube-api-access-mntqk\") pod \"maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm\" (UID: \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" Jul 08 19:43:38.364010 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.363951 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:38.364976 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.364958 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:38.404950 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.404920 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" Jul 08 19:43:38.550506 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.550415 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm"] Jul 08 19:43:38.552673 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:43:38.552645 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee4c810_d74a_486f_be94_90f84b5b8ae4.slice/crio-d6b8810da5610d69f0141d204fa2e8f8c10fd741672b41443e9494ffbaef32bf WatchSource:0}: Error finding container d6b8810da5610d69f0141d204fa2e8f8c10fd741672b41443e9494ffbaef32bf: Status 404 returned error can't find the container with id d6b8810da5610d69f0141d204fa2e8f8c10fd741672b41443e9494ffbaef32bf Jul 08 19:43:38.554158 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.554141 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:43:38.625190 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.625112 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:38.725498 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:38.725447 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:39.369550 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:39.369459 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" event={"ID":"8ee4c810-d74a-486f-be94-90f84b5b8ae4","Type":"ContainerStarted","Data":"1d5599c5417abeeea7c2cc47431f55684982629ea2961a1ab7e7a8be57821387"} Jul 08 19:43:39.369550 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:39.369526 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" event={"ID":"8ee4c810-d74a-486f-be94-90f84b5b8ae4","Type":"ContainerStarted","Data":"d6b8810da5610d69f0141d204fa2e8f8c10fd741672b41443e9494ffbaef32bf"} Jul 08 19:43:39.386714 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:39.386669 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" podStartSLOduration=0.894147527 podStartE2EDuration="1.386653545s" totalImagesPullingTime="492.506018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:43:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:43:39.385996987 +0000 UTC m=+1928.055248677" watchObservedRunningTime="2026-07-08 19:43:39.386653545 +0000 UTC m=+1928.055905235" Jul 08 19:43:39.593150 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:39.593109 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:40.374397 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:40.374354 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" Jul 08 19:43:42.002830 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.002796 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm"] Jul 08 19:43:42.030789 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.030745 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:42.130398 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.130341 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:42.216576 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.216541 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf"] Jul 08 19:43:42.216752 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.216730 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" podUID="700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" containerName="istio-proxy" containerID="cri-o://b227abdcf2e60efde91318bda1efe8cff5f16619092abb1f4317b3a97ca77173" gracePeriod=30 Jul 08 19:43:42.244432 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.244390 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:42.355213 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.355172 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:42.383713 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.383669 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" podUID="8ee4c810-d74a-486f-be94-90f84b5b8ae4" containerName="maas-api" containerID="cri-o://1d5599c5417abeeea7c2cc47431f55684982629ea2961a1ab7e7a8be57821387" gracePeriod=30 Jul 08 19:43:42.665515 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.665487 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" Jul 08 19:43:42.762401 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.762356 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ee4c810-d74a-486f-be94-90f84b5b8ae4-maas-api-tls\") pod \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\" (UID: \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\") " Jul 08 19:43:42.762582 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.762442 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mntqk\" (UniqueName: \"kubernetes.io/projected/8ee4c810-d74a-486f-be94-90f84b5b8ae4-kube-api-access-mntqk\") pod \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\" (UID: \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\") " Jul 08 19:43:42.762582 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.762462 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ee4c810-d74a-486f-be94-90f84b5b8ae4-openshift-service-ca\") pod \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\" (UID: \"8ee4c810-d74a-486f-be94-90f84b5b8ae4\") " Jul 08 19:43:42.762842 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.762810 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ee4c810-d74a-486f-be94-90f84b5b8ae4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8ee4c810-d74a-486f-be94-90f84b5b8ae4" (UID: "8ee4c810-d74a-486f-be94-90f84b5b8ae4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:43:42.764573 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.764546 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ee4c810-d74a-486f-be94-90f84b5b8ae4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8ee4c810-d74a-486f-be94-90f84b5b8ae4" (UID: "8ee4c810-d74a-486f-be94-90f84b5b8ae4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:43:42.764661 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.764605 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ee4c810-d74a-486f-be94-90f84b5b8ae4-kube-api-access-mntqk" (OuterVolumeSpecName: "kube-api-access-mntqk") pod "8ee4c810-d74a-486f-be94-90f84b5b8ae4" (UID: "8ee4c810-d74a-486f-be94-90f84b5b8ae4"). InnerVolumeSpecName "kube-api-access-mntqk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:43:42.863243 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.863207 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ee4c810-d74a-486f-be94-90f84b5b8ae4-maas-api-tls\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:43:42.863243 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.863238 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mntqk\" (UniqueName: \"kubernetes.io/projected/8ee4c810-d74a-486f-be94-90f84b5b8ae4-kube-api-access-mntqk\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:43:42.863243 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:42.863250 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ee4c810-d74a-486f-be94-90f84b5b8ae4-openshift-service-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:43:43.389744 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:43.389708 2563 generic.go:356] "Generic (PLEG): container finished" podID="8ee4c810-d74a-486f-be94-90f84b5b8ae4" containerID="1d5599c5417abeeea7c2cc47431f55684982629ea2961a1ab7e7a8be57821387" exitCode=0 Jul 08 19:43:43.390199 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:43.389759 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" Jul 08 19:43:43.390199 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:43.389801 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" event={"ID":"8ee4c810-d74a-486f-be94-90f84b5b8ae4","Type":"ContainerDied","Data":"1d5599c5417abeeea7c2cc47431f55684982629ea2961a1ab7e7a8be57821387"} Jul 08 19:43:43.390199 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:43.389859 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm" event={"ID":"8ee4c810-d74a-486f-be94-90f84b5b8ae4","Type":"ContainerDied","Data":"d6b8810da5610d69f0141d204fa2e8f8c10fd741672b41443e9494ffbaef32bf"} Jul 08 19:43:43.390199 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:43.389885 2563 scope.go:117] "RemoveContainer" containerID="1d5599c5417abeeea7c2cc47431f55684982629ea2961a1ab7e7a8be57821387" Jul 08 19:43:43.399438 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:43.399422 2563 scope.go:117] "RemoveContainer" containerID="1d5599c5417abeeea7c2cc47431f55684982629ea2961a1ab7e7a8be57821387" Jul 08 19:43:43.399701 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:43:43.399683 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d5599c5417abeeea7c2cc47431f55684982629ea2961a1ab7e7a8be57821387\": container with ID starting with 1d5599c5417abeeea7c2cc47431f55684982629ea2961a1ab7e7a8be57821387 not found: ID does not exist" containerID="1d5599c5417abeeea7c2cc47431f55684982629ea2961a1ab7e7a8be57821387" Jul 08 19:43:43.399762 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:43.399709 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d5599c5417abeeea7c2cc47431f55684982629ea2961a1ab7e7a8be57821387"} err="failed to get container status \"1d5599c5417abeeea7c2cc47431f55684982629ea2961a1ab7e7a8be57821387\": rpc error: code = NotFound desc = could not find container \"1d5599c5417abeeea7c2cc47431f55684982629ea2961a1ab7e7a8be57821387\": container with ID starting with 1d5599c5417abeeea7c2cc47431f55684982629ea2961a1ab7e7a8be57821387 not found: ID does not exist" Jul 08 19:43:43.412533 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:43.412513 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm"] Jul 08 19:43:43.416746 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:43.416727 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e2758a2d-6489fd9c4d-8hdgm"] Jul 08 19:43:43.687238 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:43.687199 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:43.903795 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:43.903760 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ee4c810-d74a-486f-be94-90f84b5b8ae4" path="/var/lib/kubelet/pods/8ee4c810-d74a-486f-be94-90f84b5b8ae4/volumes" Jul 08 19:43:47.865697 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:47.865670 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:48.013463 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.013432 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-credential-socket\") pod \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " Jul 08 19:43:48.013636 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.013479 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkjtc\" (UniqueName: \"kubernetes.io/projected/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-kube-api-access-bkjtc\") pod \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " Jul 08 19:43:48.013636 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.013510 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-envoy\") pod \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " Jul 08 19:43:48.013636 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.013543 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-podinfo\") pod \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " Jul 08 19:43:48.013636 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.013567 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istiod-ca-cert\") pod \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " Jul 08 19:43:48.013636 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.013607 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-workload-socket\") pod \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " Jul 08 19:43:48.013830 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.013656 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-data\") pod \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " Jul 08 19:43:48.013830 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.013714 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-token\") pod \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " Jul 08 19:43:48.013830 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.013735 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-workload-certs\") pod \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\" (UID: \"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae\") " Jul 08 19:43:48.013830 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.013755 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" (UID: "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:43:48.014318 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.013999 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-credential-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:43:48.014318 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.014026 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" (UID: "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:43:48.014318 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.014202 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" (UID: "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:43:48.014318 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.014266 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" (UID: "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:43:48.014614 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.014534 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-data" (OuterVolumeSpecName: "istio-data") pod "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" (UID: "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:43:48.015950 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.015923 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" (UID: "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:43:48.015950 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.015944 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-kube-api-access-bkjtc" (OuterVolumeSpecName: "kube-api-access-bkjtc") pod "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" (UID: "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae"). InnerVolumeSpecName "kube-api-access-bkjtc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:43:48.016128 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.015997 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" (UID: "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:43:48.016209 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.016195 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" (UID: "700f081f-d65f-41cd-b8ff-8ad7d77ac6ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:43:48.115127 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.115088 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-token\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:43:48.115127 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.115122 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-workload-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:43:48.115127 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.115132 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkjtc\" (UniqueName: \"kubernetes.io/projected/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-kube-api-access-bkjtc\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:43:48.115351 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.115142 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-envoy\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:43:48.115351 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.115153 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-podinfo\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:43:48.115351 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.115160 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istiod-ca-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:43:48.115351 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.115169 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-workload-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:43:48.115351 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.115176 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae-istio-data\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:43:48.195873 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.195836 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:48.294565 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.294468 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:48.414232 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.414193 2563 generic.go:356] "Generic (PLEG): container finished" podID="700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" containerID="b227abdcf2e60efde91318bda1efe8cff5f16619092abb1f4317b3a97ca77173" exitCode=0 Jul 08 19:43:48.414411 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.414239 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" Jul 08 19:43:48.414411 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.414238 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" event={"ID":"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae","Type":"ContainerDied","Data":"b227abdcf2e60efde91318bda1efe8cff5f16619092abb1f4317b3a97ca77173"} Jul 08 19:43:48.414411 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.414351 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf" event={"ID":"700f081f-d65f-41cd-b8ff-8ad7d77ac6ae","Type":"ContainerDied","Data":"a6c7b8b1150791c44ef3026cd195de73507556235b579b256ec1f44f61c256cd"} Jul 08 19:43:48.414411 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.414393 2563 scope.go:117] "RemoveContainer" containerID="b227abdcf2e60efde91318bda1efe8cff5f16619092abb1f4317b3a97ca77173" Jul 08 19:43:48.423289 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.423271 2563 scope.go:117] "RemoveContainer" containerID="b227abdcf2e60efde91318bda1efe8cff5f16619092abb1f4317b3a97ca77173" Jul 08 19:43:48.423535 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:43:48.423515 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b227abdcf2e60efde91318bda1efe8cff5f16619092abb1f4317b3a97ca77173\": container with ID starting with b227abdcf2e60efde91318bda1efe8cff5f16619092abb1f4317b3a97ca77173 not found: ID does not exist" containerID="b227abdcf2e60efde91318bda1efe8cff5f16619092abb1f4317b3a97ca77173" Jul 08 19:43:48.423593 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.423547 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b227abdcf2e60efde91318bda1efe8cff5f16619092abb1f4317b3a97ca77173"} err="failed to get container status \"b227abdcf2e60efde91318bda1efe8cff5f16619092abb1f4317b3a97ca77173\": rpc error: code = NotFound desc = could not find container \"b227abdcf2e60efde91318bda1efe8cff5f16619092abb1f4317b3a97ca77173\": container with ID starting with b227abdcf2e60efde91318bda1efe8cff5f16619092abb1f4317b3a97ca77173 not found: ID does not exist" Jul 08 19:43:48.436914 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.436888 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf"] Jul 08 19:43:48.441539 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.441521 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-e2758a2d-openshift-default-666c5fc7d7-8lfsf"] Jul 08 19:43:48.673571 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.673489 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:48.773827 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.773792 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:48.874595 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:48.874560 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:49.903799 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:49.903757 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" path="/var/lib/kubelet/pods/700f081f-d65f-41cd-b8ff-8ad7d77ac6ae/volumes" Jul 08 19:43:50.341075 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:50.341034 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:55.986095 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:55.986047 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:56.095839 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:56.095801 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:56.186572 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:56.186535 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:56.286251 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:56.286218 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:43:56.708547 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:43:56.708468 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:44:22.743671 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:44:22.743632 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:44:28.348291 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:44:28.348251 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:44:28.416474 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:44:28.416437 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:44:28.489720 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:44:28.489663 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:44:28.589620 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:44:28.589580 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:44:44.437386 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:44:44.437348 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:44:50.234364 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:44:50.234326 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:44:50.336411 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:44:50.336353 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:44:50.434435 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:44:50.434400 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:00.136303 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.136265 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9"] Jul 08 19:45:00.136776 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.136606 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" containerName="istio-proxy" Jul 08 19:45:00.136776 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.136616 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" containerName="istio-proxy" Jul 08 19:45:00.136776 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.136634 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ee4c810-d74a-486f-be94-90f84b5b8ae4" containerName="maas-api" Jul 08 19:45:00.136776 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.136640 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ee4c810-d74a-486f-be94-90f84b5b8ae4" containerName="maas-api" Jul 08 19:45:00.136776 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.136688 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ee4c810-d74a-486f-be94-90f84b5b8ae4" containerName="maas-api" Jul 08 19:45:00.136776 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.136699 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="700f081f-d65f-41cd-b8ff-8ad7d77ac6ae" containerName="istio-proxy" Jul 08 19:45:00.139696 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.139677 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9" Jul 08 19:45:00.147369 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.147241 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9"] Jul 08 19:45:00.248249 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.248215 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndq5r\" (UniqueName: \"kubernetes.io/projected/464c1964-7df7-4a60-8d1e-9a2aad9164db-kube-api-access-ndq5r\") pod \"maas-api-key-cleanup-29725665-cmxg9\" (UID: \"464c1964-7df7-4a60-8d1e-9a2aad9164db\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9" Jul 08 19:45:00.349644 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.349614 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndq5r\" (UniqueName: \"kubernetes.io/projected/464c1964-7df7-4a60-8d1e-9a2aad9164db-kube-api-access-ndq5r\") pod \"maas-api-key-cleanup-29725665-cmxg9\" (UID: \"464c1964-7df7-4a60-8d1e-9a2aad9164db\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9" Jul 08 19:45:00.358173 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.358147 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndq5r\" (UniqueName: \"kubernetes.io/projected/464c1964-7df7-4a60-8d1e-9a2aad9164db-kube-api-access-ndq5r\") pod \"maas-api-key-cleanup-29725665-cmxg9\" (UID: \"464c1964-7df7-4a60-8d1e-9a2aad9164db\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9" Jul 08 19:45:00.451078 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.451000 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9" Jul 08 19:45:00.786028 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.785999 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9"] Jul 08 19:45:00.788752 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:45:00.788725 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod464c1964_7df7_4a60_8d1e_9a2aad9164db.slice/crio-f113e02e0a299f2649ceb54a0f89864ffe38c24993396b66e2856683e6a00c59 WatchSource:0}: Error finding container f113e02e0a299f2649ceb54a0f89864ffe38c24993396b66e2856683e6a00c59: Status 404 returned error can't find the container with id f113e02e0a299f2649ceb54a0f89864ffe38c24993396b66e2856683e6a00c59 Jul 08 19:45:00.986159 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:00.986120 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:01.733813 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:01.733775 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9" event={"ID":"464c1964-7df7-4a60-8d1e-9a2aad9164db","Type":"ContainerStarted","Data":"f113e02e0a299f2649ceb54a0f89864ffe38c24993396b66e2856683e6a00c59"} Jul 08 19:45:01.820393 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:01.820339 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:04.748635 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:04.748595 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9" event={"ID":"464c1964-7df7-4a60-8d1e-9a2aad9164db","Type":"ContainerStarted","Data":"9e8ccbdbfd0cba51eab33c7ecf1c156c4190ba58f3d4dec5ba21a2889d103839"} Jul 08 19:45:04.765598 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:04.765550 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9" podStartSLOduration=1.9915283559999999 podStartE2EDuration="4.765535573s" totalImagesPullingTime="2.774007217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:45:04.763481699 +0000 UTC m=+2013.432733381" watchObservedRunningTime="2026-07-08 19:45:04.765535573 +0000 UTC m=+2013.434787263" Jul 08 19:45:07.538639 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:07.538600 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:07.599773 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:07.599716 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:07.698185 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:07.698149 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:07.798397 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:07.798277 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:07.897576 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:07.897540 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:08.928685 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:08.928649 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:09.029803 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:09.029759 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:10.529665 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:10.529625 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:16.591717 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:16.591672 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:16.696560 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:16.696519 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:17.270878 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:17.270840 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:22.958029 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:22.957979 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:23.021538 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.021505 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:23.205624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.205594 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:23.496911 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.496878 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl"] Jul 08 19:45:23.500243 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.500222 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.503088 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.503061 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d97e56e9-openshift-default-dockercfg-kklx9\"" Jul 08 19:45:23.514100 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.514077 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl"] Jul 08 19:45:23.525874 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.525846 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:23.549259 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.549231 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-credential-socket\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.549366 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.549263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istiod-ca-cert\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.549366 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.549307 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-workload-socket\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.549469 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.549416 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-envoy\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.549469 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.549460 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-workload-certs\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.549549 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.549489 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-podinfo\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.549549 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.549508 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmbdx\" (UniqueName: \"kubernetes.io/projected/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-kube-api-access-vmbdx\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.549549 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.549528 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-token\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.549697 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.549577 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-data\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.650085 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.650049 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-workload-certs\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.650281 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.650095 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-podinfo\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.650281 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.650122 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmbdx\" (UniqueName: \"kubernetes.io/projected/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-kube-api-access-vmbdx\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.650281 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.650151 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-token\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.650281 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.650194 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-data\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.650281 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.650227 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-credential-socket\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.650281 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.650256 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istiod-ca-cert\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.650634 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.650307 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-workload-socket\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.650634 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.650388 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-envoy\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.650634 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.650505 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-workload-certs\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.650634 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.650601 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-data\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.650881 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.650860 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-credential-socket\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.650946 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.650899 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-workload-socket\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.651089 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.651065 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istiod-ca-cert\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.652705 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.652685 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-podinfo\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.652784 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.652685 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-envoy\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.661101 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.661074 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-token\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.661101 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.661093 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmbdx\" (UniqueName: \"kubernetes.io/projected/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-kube-api-access-vmbdx\") pod \"e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.811597 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.811564 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:23.835762 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.835730 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:23.952207 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.952176 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl"] Jul 08 19:45:23.954914 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:45:23.954887 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02a843c4_a2bd_4dd4_994c_1655ca9a6ef8.slice/crio-739eababb6a09a8b1807e986b9913be5a32d670f8b1e24705378f32f2174b52f WatchSource:0}: Error finding container 739eababb6a09a8b1807e986b9913be5a32d670f8b1e24705378f32f2174b52f: Status 404 returned error can't find the container with id 739eababb6a09a8b1807e986b9913be5a32d670f8b1e24705378f32f2174b52f Jul 08 19:45:23.956887 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.956857 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:45:23.956950 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.956917 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:45:23.956989 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:23.956958 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:45:24.837053 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:24.837014 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" event={"ID":"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8","Type":"ContainerStarted","Data":"003fcbe571b4ff350de4ea76df3abf74a2e2a7428f61710e96e988ccc3dc3feb"} Jul 08 19:45:24.837503 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:24.837065 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" event={"ID":"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8","Type":"ContainerStarted","Data":"739eababb6a09a8b1807e986b9913be5a32d670f8b1e24705378f32f2174b52f"} Jul 08 19:45:24.857022 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:24.856974 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" podStartSLOduration=1.856960567 podStartE2EDuration="1.856960567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:45:24.856122927 +0000 UTC m=+2033.525374632" watchObservedRunningTime="2026-07-08 19:45:24.856960567 +0000 UTC m=+2033.526212256" Jul 08 19:45:25.812468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:25.812433 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:25.817389 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:25.817353 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:25.842008 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:25.841986 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:25.842952 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:25.842924 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:26.786678 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:26.786640 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:26.888993 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:26.888960 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:26.951306 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:26.951271 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6"] Jul 08 19:45:26.954964 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:26.954942 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:26.957626 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:26.957607 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d97e56e9-serving-cert\"" Jul 08 19:45:26.965128 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:26.964606 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6"] Jul 08 19:45:26.979259 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:26.979231 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-openshift-service-ca\") pod \"maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6\" (UID: \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:26.979403 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:26.979297 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhdh4\" (UniqueName: \"kubernetes.io/projected/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-kube-api-access-jhdh4\") pod \"maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6\" (UID: \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:26.979482 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:26.979432 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-maas-api-tls\") pod \"maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6\" (UID: \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:26.994633 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:26.994602 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:27.079946 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:27.079850 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-maas-api-tls\") pod \"maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6\" (UID: \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:27.079946 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:27.079910 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-openshift-service-ca\") pod \"maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6\" (UID: \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:27.079946 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:27.079949 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhdh4\" (UniqueName: \"kubernetes.io/projected/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-kube-api-access-jhdh4\") pod \"maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6\" (UID: \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:27.080248 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:45:27.080058 2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-serving-cert: secret "maas-api-e2e-mt-d97e56e9-serving-cert" not found Jul 08 19:45:27.080248 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:45:27.080140 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-maas-api-tls podName:c14eb21c-c09d-45ed-b63e-dd566cd5a33c nodeName:}" failed. No retries permitted until 2026-07-08 19:45:27.580116919 +0000 UTC m=+2036.249368594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-maas-api-tls") pod "maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" (UID: "c14eb21c-c09d-45ed-b63e-dd566cd5a33c") : secret "maas-api-e2e-mt-d97e56e9-serving-cert" not found Jul 08 19:45:27.080600 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:27.080581 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-openshift-service-ca\") pod \"maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6\" (UID: \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:27.094332 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:27.094305 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhdh4\" (UniqueName: \"kubernetes.io/projected/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-kube-api-access-jhdh4\") pod \"maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6\" (UID: \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:27.347018 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:27.346901 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:27.540142 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:27.540106 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:27.582855 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:27.582819 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-maas-api-tls\") pod \"maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6\" (UID: \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:27.585249 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:27.585218 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-maas-api-tls\") pod \"maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6\" (UID: \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:27.642334 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:27.642256 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:27.870127 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:27.870095 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:27.999660 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:27.999632 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6"] Jul 08 19:45:28.001444 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:45:28.001408 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc14eb21c_c09d_45ed_b63e_dd566cd5a33c.slice/crio-d6a9684f88bff9da65f0be4d21adc14b32671c18e3abce3bf809c6c8a6aa5b11 WatchSource:0}: Error finding container d6a9684f88bff9da65f0be4d21adc14b32671c18e3abce3bf809c6c8a6aa5b11: Status 404 returned error can't find the container with id d6a9684f88bff9da65f0be4d21adc14b32671c18e3abce3bf809c6c8a6aa5b11 Jul 08 19:45:28.860131 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:28.860102 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" event={"ID":"c14eb21c-c09d-45ed-b63e-dd566cd5a33c","Type":"ContainerStarted","Data":"0ed3d2e065dcc9b14c6a5d48b19cb3e9df479d81f89c6295ea7390bda0076679"} Jul 08 19:45:28.860300 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:28.860150 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" event={"ID":"c14eb21c-c09d-45ed-b63e-dd566cd5a33c","Type":"ContainerStarted","Data":"d6a9684f88bff9da65f0be4d21adc14b32671c18e3abce3bf809c6c8a6aa5b11"} Jul 08 19:45:28.860300 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:28.860219 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:28.878491 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:28.878443 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" podStartSLOduration=2.24357495 podStartE2EDuration="2.878429117s" totalImagesPullingTime="634.854167ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:45:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:45:28.877986047 +0000 UTC m=+2037.547237748" watchObservedRunningTime="2026-07-08 19:45:28.878429117 +0000 UTC m=+2037.547680813" Jul 08 19:45:31.288349 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:31.288316 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:31.974411 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:31.974363 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:32.072031 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:32.072000 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:32.530349 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:32.530297 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:34.870905 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:34.870871 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:38.202418 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.202353 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:38.301621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.301564 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:38.608310 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.608278 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6"] Jul 08 19:45:38.608811 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.608777 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" podUID="c14eb21c-c09d-45ed-b63e-dd566cd5a33c" containerName="maas-api" containerID="cri-o://0ed3d2e065dcc9b14c6a5d48b19cb3e9df479d81f89c6295ea7390bda0076679" gracePeriod=30 Jul 08 19:45:38.638865 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.638834 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:38.714533 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.714417 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:38.859901 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.859840 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:38.908697 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.908667 2563 generic.go:356] "Generic (PLEG): container finished" podID="c14eb21c-c09d-45ed-b63e-dd566cd5a33c" containerID="0ed3d2e065dcc9b14c6a5d48b19cb3e9df479d81f89c6295ea7390bda0076679" exitCode=0 Jul 08 19:45:38.908861 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.908738 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" Jul 08 19:45:38.908861 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.908745 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" event={"ID":"c14eb21c-c09d-45ed-b63e-dd566cd5a33c","Type":"ContainerDied","Data":"0ed3d2e065dcc9b14c6a5d48b19cb3e9df479d81f89c6295ea7390bda0076679"} Jul 08 19:45:38.908990 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.908876 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6" event={"ID":"c14eb21c-c09d-45ed-b63e-dd566cd5a33c","Type":"ContainerDied","Data":"d6a9684f88bff9da65f0be4d21adc14b32671c18e3abce3bf809c6c8a6aa5b11"} Jul 08 19:45:38.908990 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.908910 2563 scope.go:117] "RemoveContainer" containerID="0ed3d2e065dcc9b14c6a5d48b19cb3e9df479d81f89c6295ea7390bda0076679" Jul 08 19:45:38.917277 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.917240 2563 scope.go:117] "RemoveContainer" containerID="0ed3d2e065dcc9b14c6a5d48b19cb3e9df479d81f89c6295ea7390bda0076679" Jul 08 19:45:38.917566 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:45:38.917542 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ed3d2e065dcc9b14c6a5d48b19cb3e9df479d81f89c6295ea7390bda0076679\": container with ID starting with 0ed3d2e065dcc9b14c6a5d48b19cb3e9df479d81f89c6295ea7390bda0076679 not found: ID does not exist" containerID="0ed3d2e065dcc9b14c6a5d48b19cb3e9df479d81f89c6295ea7390bda0076679" Jul 08 19:45:38.917640 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.917578 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ed3d2e065dcc9b14c6a5d48b19cb3e9df479d81f89c6295ea7390bda0076679"} err="failed to get container status \"0ed3d2e065dcc9b14c6a5d48b19cb3e9df479d81f89c6295ea7390bda0076679\": rpc error: code = NotFound desc = could not find container \"0ed3d2e065dcc9b14c6a5d48b19cb3e9df479d81f89c6295ea7390bda0076679\": container with ID starting with 0ed3d2e065dcc9b14c6a5d48b19cb3e9df479d81f89c6295ea7390bda0076679 not found: ID does not exist" Jul 08 19:45:38.998276 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.998241 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-openshift-service-ca\") pod \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\" (UID: \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\") " Jul 08 19:45:38.998468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.998305 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhdh4\" (UniqueName: \"kubernetes.io/projected/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-kube-api-access-jhdh4\") pod \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\" (UID: \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\") " Jul 08 19:45:38.998468 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.998352 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-maas-api-tls\") pod \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\" (UID: \"c14eb21c-c09d-45ed-b63e-dd566cd5a33c\") " Jul 08 19:45:38.998717 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:38.998686 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c14eb21c-c09d-45ed-b63e-dd566cd5a33c" (UID: "c14eb21c-c09d-45ed-b63e-dd566cd5a33c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:45:39.000496 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.000474 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c14eb21c-c09d-45ed-b63e-dd566cd5a33c" (UID: "c14eb21c-c09d-45ed-b63e-dd566cd5a33c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:45:39.000702 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.000676 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-kube-api-access-jhdh4" (OuterVolumeSpecName: "kube-api-access-jhdh4") pod "c14eb21c-c09d-45ed-b63e-dd566cd5a33c" (UID: "c14eb21c-c09d-45ed-b63e-dd566cd5a33c"). InnerVolumeSpecName "kube-api-access-jhdh4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:45:39.079875 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.079838 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:39.099958 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.099920 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhdh4\" (UniqueName: \"kubernetes.io/projected/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-kube-api-access-jhdh4\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:45:39.099958 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.099950 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-maas-api-tls\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:45:39.099958 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.099967 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c14eb21c-c09d-45ed-b63e-dd566cd5a33c-openshift-service-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:45:39.230036 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.230007 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6"] Jul 08 19:45:39.233870 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.233844 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d97e56e9-84445cc758-x8wr6"] Jul 08 19:45:39.903765 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.903729 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c14eb21c-c09d-45ed-b63e-dd566cd5a33c" path="/var/lib/kubelet/pods/c14eb21c-c09d-45ed-b63e-dd566cd5a33c/volumes" Jul 08 19:45:39.914023 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.913995 2563 generic.go:356] "Generic (PLEG): container finished" podID="02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" containerID="003fcbe571b4ff350de4ea76df3abf74a2e2a7428f61710e96e988ccc3dc3feb" exitCode=0 Jul 08 19:45:39.914153 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.914076 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" event={"ID":"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8","Type":"ContainerDied","Data":"003fcbe571b4ff350de4ea76df3abf74a2e2a7428f61710e96e988ccc3dc3feb"} Jul 08 19:45:39.914508 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.914492 2563 scope.go:117] "RemoveContainer" containerID="003fcbe571b4ff350de4ea76df3abf74a2e2a7428f61710e96e988ccc3dc3feb" Jul 08 19:45:39.915580 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.915552 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:45:39.915650 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.915637 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:45:39.915709 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:39.915694 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:45:40.842518 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:40.842416 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:40.920771 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:40.920741 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" event={"ID":"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8","Type":"ContainerStarted","Data":"1bb2e7250f0e105d1f09cb49de7a85267c1059fd0382ba098111af65b046b185"} Jul 08 19:45:41.812135 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:41.812088 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:41.813423 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:41.813398 2563 patch_prober.go:28] interesting pod/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 08 19:45:41.813532 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:41.813464 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" podUID="02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 08 19:45:42.812601 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:42.812562 2563 patch_prober.go:28] interesting pod/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 08 19:45:42.813004 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:42.812617 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" podUID="02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 08 19:45:43.812702 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:43.812663 2563 patch_prober.go:28] interesting pod/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 08 19:45:43.813092 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:43.812728 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" podUID="02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 08 19:45:44.812769 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:44.812728 2563 patch_prober.go:28] interesting pod/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 08 19:45:44.813189 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:44.812784 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" podUID="02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 08 19:45:45.015027 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:45.014983 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:45.085144 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:45.085042 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:45.263579 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:45.263543 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl"] Jul 08 19:45:45.263854 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:45.263826 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" podUID="02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" containerName="istio-proxy" containerID="cri-o://1bb2e7250f0e105d1f09cb49de7a85267c1059fd0382ba098111af65b046b185" gracePeriod=30 Jul 08 19:45:45.279233 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:45.279203 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:45.651899 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:45.651864 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:45.749026 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:45.748990 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:45.883212 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:45.883182 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:50.519817 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.519787 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:50.605033 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.604955 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-workload-socket\") pod \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " Jul 08 19:45:50.605033 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605005 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmbdx\" (UniqueName: \"kubernetes.io/projected/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-kube-api-access-vmbdx\") pod \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " Jul 08 19:45:50.605033 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605034 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-workload-certs\") pod \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " Jul 08 19:45:50.605312 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605049 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-token\") pod \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " Jul 08 19:45:50.605312 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605093 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-credential-socket\") pod \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " Jul 08 19:45:50.605312 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605108 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-data\") pod \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " Jul 08 19:45:50.605312 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605142 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-envoy\") pod \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " Jul 08 19:45:50.605312 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605183 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istiod-ca-cert\") pod \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " Jul 08 19:45:50.605312 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605208 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-podinfo\") pod \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\" (UID: \"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8\") " Jul 08 19:45:50.605312 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605260 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" (UID: "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:50.605729 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605403 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" (UID: "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:50.605729 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605417 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" (UID: "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:50.605729 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605564 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-workload-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:45:50.605729 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605615 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-workload-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:45:50.605729 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605630 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-credential-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:45:50.606049 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.605726 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" (UID: "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:45:50.606111 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.606059 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-data" (OuterVolumeSpecName: "istio-data") pod "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" (UID: "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:50.607326 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.607292 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-kube-api-access-vmbdx" (OuterVolumeSpecName: "kube-api-access-vmbdx") pod "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" (UID: "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8"). InnerVolumeSpecName "kube-api-access-vmbdx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:45:50.607456 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.607332 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-token" (OuterVolumeSpecName: "istio-token") pod "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" (UID: "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:45:50.607669 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.607646 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" (UID: "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:50.607725 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.607664 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" (UID: "02a843c4-a2bd-4dd4-994c-1655ca9a6ef8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:45:50.706764 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.706726 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-envoy\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:45:50.706764 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.706759 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istiod-ca-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:45:50.706764 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.706769 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-podinfo\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:45:50.706764 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.706778 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmbdx\" (UniqueName: \"kubernetes.io/projected/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-kube-api-access-vmbdx\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:45:50.707003 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.706788 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-token\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:45:50.707003 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.706796 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8-istio-data\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:45:50.964596 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.964502 2563 generic.go:356] "Generic (PLEG): container finished" podID="02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" containerID="1bb2e7250f0e105d1f09cb49de7a85267c1059fd0382ba098111af65b046b185" exitCode=0 Jul 08 19:45:50.964596 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.964557 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" Jul 08 19:45:50.964775 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.964587 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" event={"ID":"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8","Type":"ContainerDied","Data":"1bb2e7250f0e105d1f09cb49de7a85267c1059fd0382ba098111af65b046b185"} Jul 08 19:45:50.964775 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.964644 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl" event={"ID":"02a843c4-a2bd-4dd4-994c-1655ca9a6ef8","Type":"ContainerDied","Data":"739eababb6a09a8b1807e986b9913be5a32d670f8b1e24705378f32f2174b52f"} Jul 08 19:45:50.964775 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.964665 2563 scope.go:117] "RemoveContainer" containerID="1bb2e7250f0e105d1f09cb49de7a85267c1059fd0382ba098111af65b046b185" Jul 08 19:45:50.973279 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.973262 2563 scope.go:117] "RemoveContainer" containerID="003fcbe571b4ff350de4ea76df3abf74a2e2a7428f61710e96e988ccc3dc3feb" Jul 08 19:45:50.981277 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.981258 2563 scope.go:117] "RemoveContainer" containerID="1bb2e7250f0e105d1f09cb49de7a85267c1059fd0382ba098111af65b046b185" Jul 08 19:45:50.981577 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:45:50.981544 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bb2e7250f0e105d1f09cb49de7a85267c1059fd0382ba098111af65b046b185\": container with ID starting with 1bb2e7250f0e105d1f09cb49de7a85267c1059fd0382ba098111af65b046b185 not found: ID does not exist" containerID="1bb2e7250f0e105d1f09cb49de7a85267c1059fd0382ba098111af65b046b185" Jul 08 19:45:50.981622 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.981585 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bb2e7250f0e105d1f09cb49de7a85267c1059fd0382ba098111af65b046b185"} err="failed to get container status \"1bb2e7250f0e105d1f09cb49de7a85267c1059fd0382ba098111af65b046b185\": rpc error: code = NotFound desc = could not find container \"1bb2e7250f0e105d1f09cb49de7a85267c1059fd0382ba098111af65b046b185\": container with ID starting with 1bb2e7250f0e105d1f09cb49de7a85267c1059fd0382ba098111af65b046b185 not found: ID does not exist" Jul 08 19:45:50.981622 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.981601 2563 scope.go:117] "RemoveContainer" containerID="003fcbe571b4ff350de4ea76df3abf74a2e2a7428f61710e96e988ccc3dc3feb" Jul 08 19:45:50.981845 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:45:50.981824 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"003fcbe571b4ff350de4ea76df3abf74a2e2a7428f61710e96e988ccc3dc3feb\": container with ID starting with 003fcbe571b4ff350de4ea76df3abf74a2e2a7428f61710e96e988ccc3dc3feb not found: ID does not exist" containerID="003fcbe571b4ff350de4ea76df3abf74a2e2a7428f61710e96e988ccc3dc3feb" Jul 08 19:45:50.981908 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.981854 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"003fcbe571b4ff350de4ea76df3abf74a2e2a7428f61710e96e988ccc3dc3feb"} err="failed to get container status \"003fcbe571b4ff350de4ea76df3abf74a2e2a7428f61710e96e988ccc3dc3feb\": rpc error: code = NotFound desc = could not find container \"003fcbe571b4ff350de4ea76df3abf74a2e2a7428f61710e96e988ccc3dc3feb\": container with ID starting with 003fcbe571b4ff350de4ea76df3abf74a2e2a7428f61710e96e988ccc3dc3feb not found: ID does not exist" Jul 08 19:45:50.986166 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.986143 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl"] Jul 08 19:45:50.992598 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:50.992578 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d97e56e9-openshift-default-df49d487f-cltzl"] Jul 08 19:45:51.603649 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:51.603619 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:51.708632 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:51.708594 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:51.803337 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:51.803298 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:51.903995 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:51.903920 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" path="/var/lib/kubelet/pods/02a843c4-a2bd-4dd4-994c-1655ca9a6ef8/volumes" Jul 08 19:45:51.955222 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:51.955183 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:57.480862 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:57.480767 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:45:57.581303 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:45:57.581265 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:00.011616 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:00.011587 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9"] Jul 08 19:46:00.012012 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:00.011770 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9" podUID="464c1964-7df7-4a60-8d1e-9a2aad9164db" containerName="cleanup" containerID="cri-o://9e8ccbdbfd0cba51eab33c7ecf1c156c4190ba58f3d4dec5ba21a2889d103839" gracePeriod=30 Jul 08 19:46:04.371864 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.371828 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:04.671975 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.671888 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp"] Jul 08 19:46:04.672367 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.672349 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" containerName="istio-proxy" Jul 08 19:46:04.672367 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.672366 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" containerName="istio-proxy" Jul 08 19:46:04.672637 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.672398 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" containerName="istio-proxy" Jul 08 19:46:04.672637 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.672407 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" containerName="istio-proxy" Jul 08 19:46:04.672637 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.672430 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c14eb21c-c09d-45ed-b63e-dd566cd5a33c" containerName="maas-api" Jul 08 19:46:04.672637 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.672438 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c14eb21c-c09d-45ed-b63e-dd566cd5a33c" containerName="maas-api" Jul 08 19:46:04.672637 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.672523 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" containerName="istio-proxy" Jul 08 19:46:04.672637 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.672534 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c14eb21c-c09d-45ed-b63e-dd566cd5a33c" containerName="maas-api" Jul 08 19:46:04.672637 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.672542 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="02a843c4-a2bd-4dd4-994c-1655ca9a6ef8" containerName="istio-proxy" Jul 08 19:46:04.675787 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.675768 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.678721 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.678697 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0dad1a0a-openshift-default-dockercfg-t7b9r\"" Jul 08 19:46:04.686789 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.686767 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp"] Jul 08 19:46:04.700000 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.699973 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:04.731606 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.731577 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-token\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.731715 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.731616 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-workload-socket\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.731715 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.731669 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-envoy\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.731805 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.731752 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t47wn\" (UniqueName: \"kubernetes.io/projected/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-kube-api-access-t47wn\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.731805 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.731795 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-podinfo\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.731897 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.731816 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istiod-ca-cert\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.731897 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.731852 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-data\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.731897 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.731873 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-workload-certs\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.732027 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.731922 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-credential-socket\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.832653 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.832613 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-credential-socket\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.832855 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.832685 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-token\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.832855 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.832722 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-workload-socket\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.832855 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.832768 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-envoy\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.832855 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.832825 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t47wn\" (UniqueName: \"kubernetes.io/projected/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-kube-api-access-t47wn\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.832855 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.832850 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-podinfo\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.833133 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.832877 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istiod-ca-cert\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.833133 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.832929 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-data\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.833133 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.832959 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-workload-certs\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.833133 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.833084 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-credential-socket\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.833351 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.833238 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-workload-socket\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.833351 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.833301 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-workload-certs\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.833487 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.833461 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-data\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.833678 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.833658 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istiod-ca-cert\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.835306 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.835285 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-envoy\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.835504 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.835485 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-podinfo\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.842101 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.842072 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t47wn\" (UniqueName: \"kubernetes.io/projected/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-kube-api-access-t47wn\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.842243 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.842228 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-token\") pod \"e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:04.879420 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.879365 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:04.986308 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:04.986229 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:05.124210 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:05.124185 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp"] Jul 08 19:46:05.126254 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:46:05.126221 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ab0056_07f1_4a8e_a371_8a5ac6c3caae.slice/crio-5a42cd2b855cb4d4b3675e68c3c74a0418a8d4014bad745cb2d4f09f4ec88dce WatchSource:0}: Error finding container 5a42cd2b855cb4d4b3675e68c3c74a0418a8d4014bad745cb2d4f09f4ec88dce: Status 404 returned error can't find the container with id 5a42cd2b855cb4d4b3675e68c3c74a0418a8d4014bad745cb2d4f09f4ec88dce Jul 08 19:46:05.128470 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:05.128436 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:46:05.128569 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:05.128508 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:46:05.128569 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:05.128540 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:46:06.038658 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:06.038616 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" event={"ID":"77ab0056-07f1-4a8e-a371-8a5ac6c3caae","Type":"ContainerStarted","Data":"99b6eb87e38ebb29bf0d1eeeb433660340621a61efc221fa99c99f0a52260d65"} Jul 08 19:46:06.038658 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:06.038665 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" event={"ID":"77ab0056-07f1-4a8e-a371-8a5ac6c3caae","Type":"ContainerStarted","Data":"5a42cd2b855cb4d4b3675e68c3c74a0418a8d4014bad745cb2d4f09f4ec88dce"} Jul 08 19:46:06.059350 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:06.059286 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" podStartSLOduration=2.059266597 podStartE2EDuration="2.059266597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:46:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:06.058856536 +0000 UTC m=+2074.728108228" watchObservedRunningTime="2026-07-08 19:46:06.059266597 +0000 UTC m=+2074.728518303" Jul 08 19:46:06.987469 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:06.987432 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:06.992430 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:06.992405 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:07.043654 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:07.043622 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:07.044655 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:07.044637 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:46:07.959217 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:07.959159 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:08.070863 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:08.070829 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:12.497494 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:12.497463 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:46:30.149865 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:30.149828 2563 generic.go:356] "Generic (PLEG): container finished" podID="464c1964-7df7-4a60-8d1e-9a2aad9164db" containerID="9e8ccbdbfd0cba51eab33c7ecf1c156c4190ba58f3d4dec5ba21a2889d103839" exitCode=137 Jul 08 19:46:30.150325 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:30.149907 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9" event={"ID":"464c1964-7df7-4a60-8d1e-9a2aad9164db","Type":"ContainerDied","Data":"9e8ccbdbfd0cba51eab33c7ecf1c156c4190ba58f3d4dec5ba21a2889d103839"} Jul 08 19:46:30.202595 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:30.202571 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9" Jul 08 19:46:30.370294 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:30.370206 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndq5r\" (UniqueName: \"kubernetes.io/projected/464c1964-7df7-4a60-8d1e-9a2aad9164db-kube-api-access-ndq5r\") pod \"464c1964-7df7-4a60-8d1e-9a2aad9164db\" (UID: \"464c1964-7df7-4a60-8d1e-9a2aad9164db\") " Jul 08 19:46:30.372254 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:30.372228 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/464c1964-7df7-4a60-8d1e-9a2aad9164db-kube-api-access-ndq5r" (OuterVolumeSpecName: "kube-api-access-ndq5r") pod "464c1964-7df7-4a60-8d1e-9a2aad9164db" (UID: "464c1964-7df7-4a60-8d1e-9a2aad9164db"). InnerVolumeSpecName "kube-api-access-ndq5r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:46:30.471642 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:30.471609 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndq5r\" (UniqueName: \"kubernetes.io/projected/464c1964-7df7-4a60-8d1e-9a2aad9164db-kube-api-access-ndq5r\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:46:31.155251 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:31.155221 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9" Jul 08 19:46:31.155251 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:31.155247 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9" event={"ID":"464c1964-7df7-4a60-8d1e-9a2aad9164db","Type":"ContainerDied","Data":"f113e02e0a299f2649ceb54a0f89864ffe38c24993396b66e2856683e6a00c59"} Jul 08 19:46:31.155737 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:31.155313 2563 scope.go:117] "RemoveContainer" containerID="9e8ccbdbfd0cba51eab33c7ecf1c156c4190ba58f3d4dec5ba21a2889d103839" Jul 08 19:46:31.177055 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:31.177022 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9"] Jul 08 19:46:31.181428 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:31.181402 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725665-cmxg9"] Jul 08 19:46:31.897858 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:31.897825 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:46:31.904252 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:31.904219 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="464c1964-7df7-4a60-8d1e-9a2aad9164db" path="/var/lib/kubelet/pods/464c1964-7df7-4a60-8d1e-9a2aad9164db/volumes" Jul 08 19:46:31.904586 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:46:31.904569 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:47:43.009180 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:47:43.009146 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:06.896751 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:06.896632 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:17.314735 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:17.314701 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:17.721272 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:17.721180 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:23.515961 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:23.515928 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:23.576696 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:23.576667 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:23.825592 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:23.825499 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp"] Jul 08 19:49:23.825811 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:23.825787 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" podUID="77ab0056-07f1-4a8e-a371-8a5ac6c3caae" containerName="istio-proxy" containerID="cri-o://99b6eb87e38ebb29bf0d1eeeb433660340621a61efc221fa99c99f0a52260d65" gracePeriod=30 Jul 08 19:49:23.860860 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:23.860823 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:25.158269 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:25.158230 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:25.257164 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:25.257119 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:25.358214 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:25.358184 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:26.039041 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:26.039009 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:26.139245 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:26.139211 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:26.239209 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:26.239170 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:27.584650 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:27.584563 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:29.084221 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.084154 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:49:29.158705 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.158674 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-workload-certs\") pod \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " Jul 08 19:49:29.158874 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.158724 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-workload-socket\") pod \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " Jul 08 19:49:29.158874 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.158743 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-envoy\") pod \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " Jul 08 19:49:29.158874 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.158772 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-podinfo\") pod \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " Jul 08 19:49:29.158874 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.158795 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-data\") pod \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " Jul 08 19:49:29.158874 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.158813 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-credential-socket\") pod \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " Jul 08 19:49:29.158874 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.158852 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istiod-ca-cert\") pod \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " Jul 08 19:49:29.159201 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.158881 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t47wn\" (UniqueName: \"kubernetes.io/projected/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-kube-api-access-t47wn\") pod \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " Jul 08 19:49:29.159201 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.158920 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-token\") pod \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\" (UID: \"77ab0056-07f1-4a8e-a371-8a5ac6c3caae\") " Jul 08 19:49:29.159201 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.158975 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "77ab0056-07f1-4a8e-a371-8a5ac6c3caae" (UID: "77ab0056-07f1-4a8e-a371-8a5ac6c3caae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:49:29.159201 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.159010 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "77ab0056-07f1-4a8e-a371-8a5ac6c3caae" (UID: "77ab0056-07f1-4a8e-a371-8a5ac6c3caae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:49:29.159201 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.159107 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "77ab0056-07f1-4a8e-a371-8a5ac6c3caae" (UID: "77ab0056-07f1-4a8e-a371-8a5ac6c3caae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:49:29.159474 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.159284 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-workload-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:49:29.159474 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.159305 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-credential-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:49:29.159474 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.159319 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-workload-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:49:29.159474 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.159351 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-data" (OuterVolumeSpecName: "istio-data") pod "77ab0056-07f1-4a8e-a371-8a5ac6c3caae" (UID: "77ab0056-07f1-4a8e-a371-8a5ac6c3caae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:49:29.159627 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.159544 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "77ab0056-07f1-4a8e-a371-8a5ac6c3caae" (UID: "77ab0056-07f1-4a8e-a371-8a5ac6c3caae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:49:29.161129 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.161100 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-kube-api-access-t47wn" (OuterVolumeSpecName: "kube-api-access-t47wn") pod "77ab0056-07f1-4a8e-a371-8a5ac6c3caae" (UID: "77ab0056-07f1-4a8e-a371-8a5ac6c3caae"). InnerVolumeSpecName "kube-api-access-t47wn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:49:29.161129 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.161112 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-token" (OuterVolumeSpecName: "istio-token") pod "77ab0056-07f1-4a8e-a371-8a5ac6c3caae" (UID: "77ab0056-07f1-4a8e-a371-8a5ac6c3caae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:49:29.161358 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.161335 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "77ab0056-07f1-4a8e-a371-8a5ac6c3caae" (UID: "77ab0056-07f1-4a8e-a371-8a5ac6c3caae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:49:29.161462 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.161429 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "77ab0056-07f1-4a8e-a371-8a5ac6c3caae" (UID: "77ab0056-07f1-4a8e-a371-8a5ac6c3caae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:49:29.259762 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.259717 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istiod-ca-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:49:29.259762 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.259756 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t47wn\" (UniqueName: \"kubernetes.io/projected/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-kube-api-access-t47wn\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:49:29.259762 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.259768 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-token\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:49:29.259762 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.259778 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-envoy\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:49:29.260023 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.259786 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-podinfo\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:49:29.260023 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.259794 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ab0056-07f1-4a8e-a371-8a5ac6c3caae-istio-data\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:49:29.940336 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.940298 2563 generic.go:356] "Generic (PLEG): container finished" podID="77ab0056-07f1-4a8e-a371-8a5ac6c3caae" containerID="99b6eb87e38ebb29bf0d1eeeb433660340621a61efc221fa99c99f0a52260d65" exitCode=0 Jul 08 19:49:29.940509 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.940353 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" Jul 08 19:49:29.940509 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.940396 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" event={"ID":"77ab0056-07f1-4a8e-a371-8a5ac6c3caae","Type":"ContainerDied","Data":"99b6eb87e38ebb29bf0d1eeeb433660340621a61efc221fa99c99f0a52260d65"} Jul 08 19:49:29.940509 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.940458 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp" event={"ID":"77ab0056-07f1-4a8e-a371-8a5ac6c3caae","Type":"ContainerDied","Data":"5a42cd2b855cb4d4b3675e68c3c74a0418a8d4014bad745cb2d4f09f4ec88dce"} Jul 08 19:49:29.940509 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.940483 2563 scope.go:117] "RemoveContainer" containerID="99b6eb87e38ebb29bf0d1eeeb433660340621a61efc221fa99c99f0a52260d65" Jul 08 19:49:29.949057 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.948877 2563 scope.go:117] "RemoveContainer" containerID="99b6eb87e38ebb29bf0d1eeeb433660340621a61efc221fa99c99f0a52260d65" Jul 08 19:49:29.949168 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:49:29.949145 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99b6eb87e38ebb29bf0d1eeeb433660340621a61efc221fa99c99f0a52260d65\": container with ID starting with 99b6eb87e38ebb29bf0d1eeeb433660340621a61efc221fa99c99f0a52260d65 not found: ID does not exist" containerID="99b6eb87e38ebb29bf0d1eeeb433660340621a61efc221fa99c99f0a52260d65" Jul 08 19:49:29.949223 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.949177 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99b6eb87e38ebb29bf0d1eeeb433660340621a61efc221fa99c99f0a52260d65"} err="failed to get container status \"99b6eb87e38ebb29bf0d1eeeb433660340621a61efc221fa99c99f0a52260d65\": rpc error: code = NotFound desc = could not find container \"99b6eb87e38ebb29bf0d1eeeb433660340621a61efc221fa99c99f0a52260d65\": container with ID starting with 99b6eb87e38ebb29bf0d1eeeb433660340621a61efc221fa99c99f0a52260d65 not found: ID does not exist" Jul 08 19:49:29.960013 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.959988 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp"] Jul 08 19:49:29.965808 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:29.965785 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0dad1a0a-openshift-default-679fb59bdc-qjkxp"] Jul 08 19:49:31.905081 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:31.905050 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77ab0056-07f1-4a8e-a371-8a5ac6c3caae" path="/var/lib/kubelet/pods/77ab0056-07f1-4a8e-a371-8a5ac6c3caae/volumes" Jul 08 19:49:33.582011 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:33.581974 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:33.652494 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:33.652455 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:34.261815 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:34.261784 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:39.909576 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:39.909538 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:40.154437 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:40.154366 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:40.233338 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:40.233203 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:40.386823 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:40.386771 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:40.534672 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:40.534626 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:50.892200 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:50.892163 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:49:52.248559 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:49:52.248503 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:12.199801 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:12.199767 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:18.291824 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:18.291786 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:18.393270 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:18.393239 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:18.705748 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:18.705662 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:19.054902 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:19.054867 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:19.393866 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:19.393734 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:22.092504 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.092455 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:22.187329 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.187281 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:22.267965 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.267927 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c"] Jul 08 19:50:22.268479 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.268450 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="464c1964-7df7-4a60-8d1e-9a2aad9164db" containerName="cleanup" Jul 08 19:50:22.268479 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.268473 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="464c1964-7df7-4a60-8d1e-9a2aad9164db" containerName="cleanup" Jul 08 19:50:22.268624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.268510 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77ab0056-07f1-4a8e-a371-8a5ac6c3caae" containerName="istio-proxy" Jul 08 19:50:22.268624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.268519 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="77ab0056-07f1-4a8e-a371-8a5ac6c3caae" containerName="istio-proxy" Jul 08 19:50:22.268624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.268598 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="77ab0056-07f1-4a8e-a371-8a5ac6c3caae" containerName="istio-proxy" Jul 08 19:50:22.268624 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.268615 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="464c1964-7df7-4a60-8d1e-9a2aad9164db" containerName="cleanup" Jul 08 19:50:22.271719 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.271700 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:50:22.274451 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.274429 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-0df657-serving-cert\"" Jul 08 19:50:22.282062 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.282040 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c"] Jul 08 19:50:22.314879 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.314847 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:22.406306 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.406214 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7d82bfe-4a2a-4828-90ea-651efdf3b170-maas-api-tls\") pod \"maas-api-e2e-inf-a-0df657-5849f86484-srb4c\" (UID: \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:50:22.406306 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.406249 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ngw7\" (UniqueName: \"kubernetes.io/projected/a7d82bfe-4a2a-4828-90ea-651efdf3b170-kube-api-access-9ngw7\") pod \"maas-api-e2e-inf-a-0df657-5849f86484-srb4c\" (UID: \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:50:22.406306 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.406274 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7d82bfe-4a2a-4828-90ea-651efdf3b170-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0df657-5849f86484-srb4c\" (UID: \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:50:22.417590 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.417560 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:22.507350 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.507324 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7d82bfe-4a2a-4828-90ea-651efdf3b170-maas-api-tls\") pod \"maas-api-e2e-inf-a-0df657-5849f86484-srb4c\" (UID: \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:50:22.507503 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.507352 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ngw7\" (UniqueName: \"kubernetes.io/projected/a7d82bfe-4a2a-4828-90ea-651efdf3b170-kube-api-access-9ngw7\") pod \"maas-api-e2e-inf-a-0df657-5849f86484-srb4c\" (UID: \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:50:22.507503 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.507410 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7d82bfe-4a2a-4828-90ea-651efdf3b170-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0df657-5849f86484-srb4c\" (UID: \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:50:22.508032 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.508013 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7d82bfe-4a2a-4828-90ea-651efdf3b170-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0df657-5849f86484-srb4c\" (UID: \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:50:22.509771 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.509750 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7d82bfe-4a2a-4828-90ea-651efdf3b170-maas-api-tls\") pod \"maas-api-e2e-inf-a-0df657-5849f86484-srb4c\" (UID: \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:50:22.521508 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.521488 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ngw7\" (UniqueName: \"kubernetes.io/projected/a7d82bfe-4a2a-4828-90ea-651efdf3b170-kube-api-access-9ngw7\") pod \"maas-api-e2e-inf-a-0df657-5849f86484-srb4c\" (UID: \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:50:22.584133 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.584108 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:50:22.728645 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.728610 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c"] Jul 08 19:50:22.730993 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:50:22.730968 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7d82bfe_4a2a_4828_90ea_651efdf3b170.slice/crio-1189f63703a2cf14959876537ad476fda191c39bb8e9deee7bcbfd1d3b6217e4 WatchSource:0}: Error finding container 1189f63703a2cf14959876537ad476fda191c39bb8e9deee7bcbfd1d3b6217e4: Status 404 returned error can't find the container with id 1189f63703a2cf14959876537ad476fda191c39bb8e9deee7bcbfd1d3b6217e4 Jul 08 19:50:22.732669 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.732651 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:50:22.902084 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:22.902052 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:23.027840 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:23.027805 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:23.172241 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:23.172199 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" event={"ID":"a7d82bfe-4a2a-4828-90ea-651efdf3b170","Type":"ContainerStarted","Data":"1189f63703a2cf14959876537ad476fda191c39bb8e9deee7bcbfd1d3b6217e4"} Jul 08 19:50:24.178072 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:24.178033 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" event={"ID":"a7d82bfe-4a2a-4828-90ea-651efdf3b170","Type":"ContainerStarted","Data":"58384c68bfa3f20a0e3eebe891d38c3b102f5782504ebd7ab2b77a991fe91981"} Jul 08 19:50:24.178501 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:24.178102 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:50:24.198525 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:24.198479 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" podStartSLOduration=1.660244764 podStartE2EDuration="2.198462469s" totalImagesPullingTime="538.217705ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:50:24.196946695 +0000 UTC m=+2332.866198379" watchObservedRunningTime="2026-07-08 19:50:24.198462469 +0000 UTC m=+2332.867714157" Jul 08 19:50:26.554905 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:26.554860 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:26.898392 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:26.898296 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:27.248358 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.248274 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642"] Jul 08 19:50:27.251701 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.251681 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.254681 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.254658 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ed6cb9-openshift-default-dockercfg-d96jh\"" Jul 08 19:50:27.267005 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.266982 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642"] Jul 08 19:50:27.291611 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.291581 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:27.352297 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.352264 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-istio-data\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.352511 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.352315 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-credential-socket\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.352511 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.352450 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-workload-socket\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.352511 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.352490 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-istio-envoy\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.352705 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.352524 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b38f3a6-3155-430e-99f3-256628991c1c-istio-podinfo\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.352705 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.352552 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b38f3a6-3155-430e-99f3-256628991c1c-istiod-ca-cert\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.352705 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.352587 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvdgj\" (UniqueName: \"kubernetes.io/projected/3b38f3a6-3155-430e-99f3-256628991c1c-kube-api-access-jvdgj\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.352818 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.352706 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b38f3a6-3155-430e-99f3-256628991c1c-istio-token\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.352818 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.352734 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-workload-certs\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.453621 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.453578 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b38f3a6-3155-430e-99f3-256628991c1c-istio-token\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.453819 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.453664 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-workload-certs\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.453819 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.453724 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-istio-data\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.453819 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.453768 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-credential-socket\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.454001 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.453819 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-workload-socket\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.454001 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.453851 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-istio-envoy\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.454001 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.453884 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b38f3a6-3155-430e-99f3-256628991c1c-istio-podinfo\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.454001 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.453912 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b38f3a6-3155-430e-99f3-256628991c1c-istiod-ca-cert\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.454001 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.453949 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvdgj\" (UniqueName: \"kubernetes.io/projected/3b38f3a6-3155-430e-99f3-256628991c1c-kube-api-access-jvdgj\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.454405 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.454362 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-workload-certs\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.454527 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.454498 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-workload-socket\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.454639 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.454415 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-credential-socket\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.454639 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.454467 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-istio-data\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.454966 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.454942 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b38f3a6-3155-430e-99f3-256628991c1c-istiod-ca-cert\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.456178 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.456156 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-istio-envoy\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.456661 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.456638 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b38f3a6-3155-430e-99f3-256628991c1c-istio-podinfo\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.462114 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.462083 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvdgj\" (UniqueName: \"kubernetes.io/projected/3b38f3a6-3155-430e-99f3-256628991c1c-kube-api-access-jvdgj\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.462264 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.462242 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b38f3a6-3155-430e-99f3-256628991c1c-istio-token\") pod \"e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.545076 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.545044 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:27.562870 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.562831 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:27.728490 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.728461 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642"] Jul 08 19:50:27.731844 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:50:27.731818 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b38f3a6_3155_430e_99f3_256628991c1c.slice/crio-1e96e43a73c7406481b0d22c7a7cbc62c530bbccbcca74a7e75d45cbe8777faf WatchSource:0}: Error finding container 1e96e43a73c7406481b0d22c7a7cbc62c530bbccbcca74a7e75d45cbe8777faf: Status 404 returned error can't find the container with id 1e96e43a73c7406481b0d22c7a7cbc62c530bbccbcca74a7e75d45cbe8777faf Jul 08 19:50:27.734144 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.734109 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:50:27.734200 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.734179 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:50:27.734241 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:27.734208 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 19:50:28.197787 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:28.197750 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" event={"ID":"3b38f3a6-3155-430e-99f3-256628991c1c","Type":"ContainerStarted","Data":"28d0ac8e055062e296c6a4c6f4e6cd6954b9fd31d0876f2aa282b7c516263c07"} Jul 08 19:50:28.197979 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:28.197805 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" event={"ID":"3b38f3a6-3155-430e-99f3-256628991c1c","Type":"ContainerStarted","Data":"1e96e43a73c7406481b0d22c7a7cbc62c530bbccbcca74a7e75d45cbe8777faf"} Jul 08 19:50:28.222969 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:28.222926 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" podStartSLOduration=1.222910261 podStartE2EDuration="1.222910261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:50:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:50:28.220884745 +0000 UTC m=+2336.890136434" watchObservedRunningTime="2026-07-08 19:50:28.222910261 +0000 UTC m=+2336.892161951" Jul 08 19:50:28.563320 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:28.563272 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:28.568360 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:28.568335 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:29.203616 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:29.203585 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:29.204791 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:29.204767 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:50:30.190082 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:30.190052 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:50:30.237290 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:30.237249 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:30.328772 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:30.328738 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:50:34.713259 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:50:34.713228 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:51:00.529288 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:51:00.529253 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:51:31.924987 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:51:31.924953 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:51:31.930519 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:51:31.930495 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:53:39.390496 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.390419 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c"] Jul 08 19:53:39.390981 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.390631 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" podUID="a7d82bfe-4a2a-4828-90ea-651efdf3b170" containerName="maas-api" containerID="cri-o://58384c68bfa3f20a0e3eebe891d38c3b102f5782504ebd7ab2b77a991fe91981" gracePeriod=30 Jul 08 19:53:39.462186 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.462152 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:39.551423 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.551364 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:39.630351 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.630323 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:53:39.659190 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.659116 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ngw7\" (UniqueName: \"kubernetes.io/projected/a7d82bfe-4a2a-4828-90ea-651efdf3b170-kube-api-access-9ngw7\") pod \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\" (UID: \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\") " Jul 08 19:53:39.659190 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.659162 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7d82bfe-4a2a-4828-90ea-651efdf3b170-maas-api-tls\") pod \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\" (UID: \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\") " Jul 08 19:53:39.659396 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.659198 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7d82bfe-4a2a-4828-90ea-651efdf3b170-openshift-service-ca\") pod \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\" (UID: \"a7d82bfe-4a2a-4828-90ea-651efdf3b170\") " Jul 08 19:53:39.659644 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.659621 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7d82bfe-4a2a-4828-90ea-651efdf3b170-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a7d82bfe-4a2a-4828-90ea-651efdf3b170" (UID: "a7d82bfe-4a2a-4828-90ea-651efdf3b170"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:53:39.661436 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.661408 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7d82bfe-4a2a-4828-90ea-651efdf3b170-kube-api-access-9ngw7" (OuterVolumeSpecName: "kube-api-access-9ngw7") pod "a7d82bfe-4a2a-4828-90ea-651efdf3b170" (UID: "a7d82bfe-4a2a-4828-90ea-651efdf3b170"). InnerVolumeSpecName "kube-api-access-9ngw7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:53:39.661538 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.661459 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7d82bfe-4a2a-4828-90ea-651efdf3b170-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a7d82bfe-4a2a-4828-90ea-651efdf3b170" (UID: "a7d82bfe-4a2a-4828-90ea-651efdf3b170"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:53:39.760224 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.760189 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ngw7\" (UniqueName: \"kubernetes.io/projected/a7d82bfe-4a2a-4828-90ea-651efdf3b170-kube-api-access-9ngw7\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:53:39.760224 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.760219 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7d82bfe-4a2a-4828-90ea-651efdf3b170-maas-api-tls\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:53:39.760224 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.760229 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7d82bfe-4a2a-4828-90ea-651efdf3b170-openshift-service-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:53:39.855007 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:39.854969 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:40.054100 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:40.054065 2563 generic.go:356] "Generic (PLEG): container finished" podID="a7d82bfe-4a2a-4828-90ea-651efdf3b170" containerID="58384c68bfa3f20a0e3eebe891d38c3b102f5782504ebd7ab2b77a991fe91981" exitCode=0 Jul 08 19:53:40.054284 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:40.054108 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" Jul 08 19:53:40.054284 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:40.054105 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" event={"ID":"a7d82bfe-4a2a-4828-90ea-651efdf3b170","Type":"ContainerDied","Data":"58384c68bfa3f20a0e3eebe891d38c3b102f5782504ebd7ab2b77a991fe91981"} Jul 08 19:53:40.054284 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:40.054217 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c" event={"ID":"a7d82bfe-4a2a-4828-90ea-651efdf3b170","Type":"ContainerDied","Data":"1189f63703a2cf14959876537ad476fda191c39bb8e9deee7bcbfd1d3b6217e4"} Jul 08 19:53:40.054284 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:40.054238 2563 scope.go:117] "RemoveContainer" containerID="58384c68bfa3f20a0e3eebe891d38c3b102f5782504ebd7ab2b77a991fe91981" Jul 08 19:53:40.062320 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:40.062302 2563 scope.go:117] "RemoveContainer" containerID="58384c68bfa3f20a0e3eebe891d38c3b102f5782504ebd7ab2b77a991fe91981" Jul 08 19:53:40.062612 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:53:40.062591 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58384c68bfa3f20a0e3eebe891d38c3b102f5782504ebd7ab2b77a991fe91981\": container with ID starting with 58384c68bfa3f20a0e3eebe891d38c3b102f5782504ebd7ab2b77a991fe91981 not found: ID does not exist" containerID="58384c68bfa3f20a0e3eebe891d38c3b102f5782504ebd7ab2b77a991fe91981" Jul 08 19:53:40.062679 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:40.062626 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58384c68bfa3f20a0e3eebe891d38c3b102f5782504ebd7ab2b77a991fe91981"} err="failed to get container status \"58384c68bfa3f20a0e3eebe891d38c3b102f5782504ebd7ab2b77a991fe91981\": rpc error: code = NotFound desc = could not find container \"58384c68bfa3f20a0e3eebe891d38c3b102f5782504ebd7ab2b77a991fe91981\": container with ID starting with 58384c68bfa3f20a0e3eebe891d38c3b102f5782504ebd7ab2b77a991fe91981 not found: ID does not exist" Jul 08 19:53:40.075993 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:40.075970 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c"] Jul 08 19:53:40.082583 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:40.082556 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0df657-5849f86484-srb4c"] Jul 08 19:53:41.907751 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:41.907719 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7d82bfe-4a2a-4828-90ea-651efdf3b170" path="/var/lib/kubelet/pods/a7d82bfe-4a2a-4828-90ea-651efdf3b170/volumes" Jul 08 19:53:46.181347 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:46.181304 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:46.305056 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:46.305018 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:46.492443 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:46.492352 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:46.972116 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:46.972084 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:47.408078 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:47.408036 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:54.336647 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:54.336597 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:54.432323 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:54.432288 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:54.694670 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:54.694585 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642"] Jul 08 19:53:54.694882 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:54.694857 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" podUID="3b38f3a6-3155-430e-99f3-256628991c1c" containerName="istio-proxy" containerID="cri-o://28d0ac8e055062e296c6a4c6f4e6cd6954b9fd31d0876f2aa282b7c516263c07" gracePeriod=30 Jul 08 19:53:54.749065 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:54.749026 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:53:59.206219 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:59.206186 2563 patch_prober.go:28] interesting pod/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 19:53:59.206638 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:59.206241 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" podUID="3b38f3a6-3155-430e-99f3-256628991c1c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 19:53:59.941784 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:53:59.941757 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:54:00.047629 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.047599 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-workload-socket\") pod \"3b38f3a6-3155-430e-99f3-256628991c1c\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " Jul 08 19:54:00.047814 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.047633 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-istio-envoy\") pod \"3b38f3a6-3155-430e-99f3-256628991c1c\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " Jul 08 19:54:00.047814 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.047676 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvdgj\" (UniqueName: \"kubernetes.io/projected/3b38f3a6-3155-430e-99f3-256628991c1c-kube-api-access-jvdgj\") pod \"3b38f3a6-3155-430e-99f3-256628991c1c\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " Jul 08 19:54:00.047814 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.047700 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-workload-certs\") pod \"3b38f3a6-3155-430e-99f3-256628991c1c\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " Jul 08 19:54:00.047814 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.047725 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-istio-data\") pod \"3b38f3a6-3155-430e-99f3-256628991c1c\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " Jul 08 19:54:00.047814 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.047786 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b38f3a6-3155-430e-99f3-256628991c1c-istio-podinfo\") pod \"3b38f3a6-3155-430e-99f3-256628991c1c\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " Jul 08 19:54:00.048077 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.047830 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b38f3a6-3155-430e-99f3-256628991c1c-istio-token\") pod \"3b38f3a6-3155-430e-99f3-256628991c1c\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " Jul 08 19:54:00.048077 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.047863 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-credential-socket\") pod \"3b38f3a6-3155-430e-99f3-256628991c1c\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " Jul 08 19:54:00.048077 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.047890 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b38f3a6-3155-430e-99f3-256628991c1c" (UID: "3b38f3a6-3155-430e-99f3-256628991c1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:54:00.048077 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.047911 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b38f3a6-3155-430e-99f3-256628991c1c-istiod-ca-cert\") pod \"3b38f3a6-3155-430e-99f3-256628991c1c\" (UID: \"3b38f3a6-3155-430e-99f3-256628991c1c\") " Jul 08 19:54:00.048077 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.048042 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b38f3a6-3155-430e-99f3-256628991c1c" (UID: "3b38f3a6-3155-430e-99f3-256628991c1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:54:00.048077 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.048053 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b38f3a6-3155-430e-99f3-256628991c1c" (UID: "3b38f3a6-3155-430e-99f3-256628991c1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:54:00.048424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.048137 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b38f3a6-3155-430e-99f3-256628991c1c" (UID: "3b38f3a6-3155-430e-99f3-256628991c1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:54:00.048424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.048275 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-workload-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:54:00.048424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.048297 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-istio-data\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:54:00.048424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.048311 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-credential-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:54:00.048424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.048326 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-workload-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:54:00.048424 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.048367 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b38f3a6-3155-430e-99f3-256628991c1c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b38f3a6-3155-430e-99f3-256628991c1c" (UID: "3b38f3a6-3155-430e-99f3-256628991c1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:54:00.050018 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.049984 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b38f3a6-3155-430e-99f3-256628991c1c-kube-api-access-jvdgj" (OuterVolumeSpecName: "kube-api-access-jvdgj") pod "3b38f3a6-3155-430e-99f3-256628991c1c" (UID: "3b38f3a6-3155-430e-99f3-256628991c1c"). InnerVolumeSpecName "kube-api-access-jvdgj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:54:00.050129 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.050086 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b38f3a6-3155-430e-99f3-256628991c1c" (UID: "3b38f3a6-3155-430e-99f3-256628991c1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:54:00.050253 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.050235 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b38f3a6-3155-430e-99f3-256628991c1c-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b38f3a6-3155-430e-99f3-256628991c1c" (UID: "3b38f3a6-3155-430e-99f3-256628991c1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:54:00.050355 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.050335 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b38f3a6-3155-430e-99f3-256628991c1c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b38f3a6-3155-430e-99f3-256628991c1c" (UID: "3b38f3a6-3155-430e-99f3-256628991c1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:54:00.148762 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.148729 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b38f3a6-3155-430e-99f3-256628991c1c-istio-podinfo\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:54:00.148762 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.148757 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b38f3a6-3155-430e-99f3-256628991c1c-istio-token\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:54:00.148762 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.148769 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b38f3a6-3155-430e-99f3-256628991c1c-istiod-ca-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:54:00.149013 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.148782 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b38f3a6-3155-430e-99f3-256628991c1c-istio-envoy\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:54:00.149013 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.148793 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvdgj\" (UniqueName: \"kubernetes.io/projected/3b38f3a6-3155-430e-99f3-256628991c1c-kube-api-access-jvdgj\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\"" Jul 08 19:54:00.151539 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.151511 2563 generic.go:356] "Generic (PLEG): container finished" podID="3b38f3a6-3155-430e-99f3-256628991c1c" containerID="28d0ac8e055062e296c6a4c6f4e6cd6954b9fd31d0876f2aa282b7c516263c07" exitCode=0 Jul 08 19:54:00.151641 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.151564 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" Jul 08 19:54:00.151641 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.151596 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" event={"ID":"3b38f3a6-3155-430e-99f3-256628991c1c","Type":"ContainerDied","Data":"28d0ac8e055062e296c6a4c6f4e6cd6954b9fd31d0876f2aa282b7c516263c07"} Jul 08 19:54:00.151719 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.151653 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642" event={"ID":"3b38f3a6-3155-430e-99f3-256628991c1c","Type":"ContainerDied","Data":"1e96e43a73c7406481b0d22c7a7cbc62c530bbccbcca74a7e75d45cbe8777faf"} Jul 08 19:54:00.151719 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.151676 2563 scope.go:117] "RemoveContainer" containerID="28d0ac8e055062e296c6a4c6f4e6cd6954b9fd31d0876f2aa282b7c516263c07" Jul 08 19:54:00.160860 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.160839 2563 scope.go:117] "RemoveContainer" containerID="28d0ac8e055062e296c6a4c6f4e6cd6954b9fd31d0876f2aa282b7c516263c07" Jul 08 19:54:00.161113 ip-10-0-128-253 kubenswrapper[2563]: E0708 19:54:00.161093 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28d0ac8e055062e296c6a4c6f4e6cd6954b9fd31d0876f2aa282b7c516263c07\": container with ID starting with 28d0ac8e055062e296c6a4c6f4e6cd6954b9fd31d0876f2aa282b7c516263c07 not found: ID does not exist" containerID="28d0ac8e055062e296c6a4c6f4e6cd6954b9fd31d0876f2aa282b7c516263c07" Jul 08 19:54:00.161174 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.161120 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28d0ac8e055062e296c6a4c6f4e6cd6954b9fd31d0876f2aa282b7c516263c07"} err="failed to get container status \"28d0ac8e055062e296c6a4c6f4e6cd6954b9fd31d0876f2aa282b7c516263c07\": rpc error: code = NotFound desc = could not find container \"28d0ac8e055062e296c6a4c6f4e6cd6954b9fd31d0876f2aa282b7c516263c07\": container with ID starting with 28d0ac8e055062e296c6a4c6f4e6cd6954b9fd31d0876f2aa282b7c516263c07 not found: ID does not exist" Jul 08 19:54:00.175615 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.175592 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642"] Jul 08 19:54:00.180103 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.180082 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ed6cb9-openshift-default-6cd9bd6b5c-d4642"] Jul 08 19:54:00.364279 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.364177 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c9c9cf5d4-x8fvq_ad50df7b-244c-4f7c-9d24-94e324fa5b35/manager/0.log" Jul 08 19:54:00.990981 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.990954 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dc4d9889-h4phf_8c2b21b5-fba4-4572-b4f6-a900073ecdec/copy-manifests/0.log" Jul 08 19:54:00.998803 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:00.998778 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dc4d9889-h4phf_8c2b21b5-fba4-4572-b4f6-a900073ecdec/manager/0.log" Jul 08 19:54:01.904885 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:01.904855 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b38f3a6-3155-430e-99f3-256628991c1c" path="/var/lib/kubelet/pods/3b38f3a6-3155-430e-99f3-256628991c1c/volumes" Jul 08 19:54:03.757994 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:03.757963 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-h89hn_0fbfa103-5cdd-4421-9061-f26746bc71a6/discovery/0.log" Jul 08 19:54:04.109992 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:04.109910 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xg7ds_1890d3fc-58bc-446b-b53f-300a7b637bab/istio-proxy/1.log" Jul 08 19:54:04.238860 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:04.238830 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-hf2nf_68c23503-4cdc-44d9-b1f1-60dbadd5793b/payload-pre-processing/0.log" Jul 08 19:54:04.370013 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:04.369932 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9nskb_1d6395bc-72fc-4973-a0ad-53909b703a42/payload-processing/0.log" Jul 08 19:54:04.970803 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:04.970768 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-rmnc6_be6655e0-b40e-4247-b9c0-f2e065b1b846/main/0.log" Jul 08 19:54:09.977565 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:09.977526 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:10.065873 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:10.065833 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:13.548603 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:13.548562 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z22h9_508e47f4-f845-4e04-88dd-1ab13d22d3b8/global-pull-secret-syncer/0.log" Jul 08 19:54:13.745745 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:13.745714 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" Jul 08 19:54:14.129755 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:14.129718 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-253.ec2.internal_691ef27feea25903bb266b1bd1efee6e/haproxy/0.log" Jul 08 19:54:21.897744 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:21.897706 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-w9db9_9e82e9f4-ccde-4b94-91f9-c5ed181ae83c/cluster-monitoring-operator/0.log" Jul 08 19:54:22.911969 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.911926 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld"] Jul 08 19:54:22.912399 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.912250 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b38f3a6-3155-430e-99f3-256628991c1c" containerName="istio-proxy" Jul 08 19:54:22.912399 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.912262 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b38f3a6-3155-430e-99f3-256628991c1c" containerName="istio-proxy" Jul 08 19:54:22.912399 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.912286 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7d82bfe-4a2a-4828-90ea-651efdf3b170" containerName="maas-api" Jul 08 19:54:22.912399 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.912292 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7d82bfe-4a2a-4828-90ea-651efdf3b170" containerName="maas-api" Jul 08 19:54:22.912399 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.912344 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b38f3a6-3155-430e-99f3-256628991c1c" containerName="istio-proxy" Jul 08 19:54:22.912399 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.912350 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7d82bfe-4a2a-4828-90ea-651efdf3b170" containerName="maas-api" Jul 08 19:54:22.915252 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.915233 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:22.918740 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.918721 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4ch4n\"/\"kube-root-ca.crt\"" Jul 08 19:54:22.918871 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.918776 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4ch4n\"/\"openshift-service-ca.crt\"" Jul 08 19:54:22.920022 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.920000 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4ch4n\"/\"default-dockercfg-ml2jp\"" Jul 08 19:54:22.937496 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.937474 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld"] Jul 08 19:54:22.937844 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.937823 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c085fc1e-cb38-44ee-837a-12de91006946-proc\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:22.937944 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.937893 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5f2k\" (UniqueName: \"kubernetes.io/projected/c085fc1e-cb38-44ee-837a-12de91006946-kube-api-access-q5f2k\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:22.938002 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.937981 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c085fc1e-cb38-44ee-837a-12de91006946-podres\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:22.938069 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.938052 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c085fc1e-cb38-44ee-837a-12de91006946-sys\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:22.938124 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:22.938083 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c085fc1e-cb38-44ee-837a-12de91006946-lib-modules\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:23.032626 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.032591 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vkmx9_aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a/node-exporter/0.log" Jul 08 19:54:23.039155 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.039129 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5f2k\" (UniqueName: \"kubernetes.io/projected/c085fc1e-cb38-44ee-837a-12de91006946-kube-api-access-q5f2k\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:23.039277 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.039172 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c085fc1e-cb38-44ee-837a-12de91006946-podres\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:23.039329 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.039281 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c085fc1e-cb38-44ee-837a-12de91006946-podres\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:23.039329 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.039288 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c085fc1e-cb38-44ee-837a-12de91006946-sys\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:23.039329 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.039321 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c085fc1e-cb38-44ee-837a-12de91006946-sys\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:23.039329 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.039324 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c085fc1e-cb38-44ee-837a-12de91006946-lib-modules\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:23.039519 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.039366 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c085fc1e-cb38-44ee-837a-12de91006946-proc\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:23.039519 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.039459 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c085fc1e-cb38-44ee-837a-12de91006946-lib-modules\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:23.039519 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.039488 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c085fc1e-cb38-44ee-837a-12de91006946-proc\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:23.067712 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.067687 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5f2k\" (UniqueName: \"kubernetes.io/projected/c085fc1e-cb38-44ee-837a-12de91006946-kube-api-access-q5f2k\") pod \"perf-node-gather-daemonset-xqrld\" (UID: \"c085fc1e-cb38-44ee-837a-12de91006946\") " pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:23.115283 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.115252 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vkmx9_aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a/kube-rbac-proxy/0.log" Jul 08 19:54:23.187564 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.187491 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vkmx9_aa8b384e-93cf-4e85-9e4b-0f0b3ff1724a/init-textfile/0.log" Jul 08 19:54:23.225351 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.225301 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:23.375880 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:23.375847 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld"] Jul 08 19:54:23.376121 ip-10-0-128-253 kubenswrapper[2563]: W0708 19:54:23.376095 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc085fc1e_cb38_44ee_837a_12de91006946.slice/crio-b8412e856a1a8217dbf7f9fff2f7eb419233313733ec0230598b2e9291d3a951 WatchSource:0}: Error finding container b8412e856a1a8217dbf7f9fff2f7eb419233313733ec0230598b2e9291d3a951: Status 404 returned error can't find the container with id b8412e856a1a8217dbf7f9fff2f7eb419233313733ec0230598b2e9291d3a951 Jul 08 19:54:24.257791 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:24.257755 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" event={"ID":"c085fc1e-cb38-44ee-837a-12de91006946","Type":"ContainerStarted","Data":"93c0fd791cd6d8bc480c2152537c06a23edd19f213c9b99cda0531fa1b53dbd9"} Jul 08 19:54:24.258196 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:24.257807 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" event={"ID":"c085fc1e-cb38-44ee-837a-12de91006946","Type":"ContainerStarted","Data":"b8412e856a1a8217dbf7f9fff2f7eb419233313733ec0230598b2e9291d3a951"} Jul 08 19:54:24.258196 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:24.257877 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:24.287832 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:24.287784 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" podStartSLOduration=2.287770093 podStartE2EDuration="2.287770093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:54:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:54:24.286197357 +0000 UTC m=+2572.955449046" watchObservedRunningTime="2026-07-08 19:54:24.287770093 +0000 UTC m=+2572.957021826" Jul 08 19:54:25.731852 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:25.731822 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-7ff64_98dcb86d-1429-4f0d-8740-0f4fa76916d5/networking-console-plugin/0.log" Jul 08 19:54:26.399508 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:26.399480 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/1.log" Jul 08 19:54:26.416532 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:26.416498 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tppst_e1b57420-0780-41af-8d72-56425fe6a28d/console-operator/2.log" Jul 08 19:54:29.093287 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:29.093247 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7bqln_51e8638f-de06-407c-9625-327c48e1157d/dns/0.log" Jul 08 19:54:29.134938 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:29.134902 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7bqln_51e8638f-de06-407c-9625-327c48e1157d/kube-rbac-proxy/0.log" Jul 08 19:54:29.537093 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:29.537063 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ppkjh_237823e9-8de2-4851-b96f-ec6cf742d306/dns-node-resolver/0.log" Jul 08 19:54:30.271722 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:30.271692 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4ch4n/perf-node-gather-daemonset-xqrld" Jul 08 19:54:30.400905 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:30.400876 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-64b964b7cc-vpq7c_fd0826c4-4f05-4da4-89ab-ccf456cc1074/registry/0.log" Jul 08 19:54:30.596281 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:30.596198 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n2kkx_352fc82d-a3e3-4157-ae41-5eade870e4f0/node-ca/0.log" Jul 08 19:54:32.194055 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:32.193963 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-h89hn_0fbfa103-5cdd-4421-9061-f26746bc71a6/discovery/0.log" Jul 08 19:54:32.576739 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:32.576708 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xg7ds_1890d3fc-58bc-446b-b53f-300a7b637bab/istio-proxy/0.log" Jul 08 19:54:32.630576 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:32.630541 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xg7ds_1890d3fc-58bc-446b-b53f-300a7b637bab/istio-proxy/1.log" Jul 08 19:54:32.852816 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:32.852707 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-hf2nf_68c23503-4cdc-44d9-b1f1-60dbadd5793b/payload-pre-processing/0.log" Jul 08 19:54:33.009144 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:33.009112 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9nskb_1d6395bc-72fc-4973-a0ad-53909b703a42/payload-processing/0.log" Jul 08 19:54:33.396058 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:33.396027 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-h89hn_0fbfa103-5cdd-4421-9061-f26746bc71a6/discovery/0.log" Jul 08 19:54:33.604526 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:33.604495 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xg7ds_1890d3fc-58bc-446b-b53f-300a7b637bab/istio-proxy/0.log" Jul 08 19:54:33.659596 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:33.659567 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xg7ds_1890d3fc-58bc-446b-b53f-300a7b637bab/istio-proxy/1.log" Jul 08 19:54:33.839209 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:33.839182 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-hf2nf_68c23503-4cdc-44d9-b1f1-60dbadd5793b/payload-pre-processing/0.log" Jul 08 19:54:33.950931 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:33.950906 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9nskb_1d6395bc-72fc-4973-a0ad-53909b703a42/payload-processing/0.log" Jul 08 19:54:34.244524 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:34.244442 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hpgnx_a8262b98-73be-424f-9ac6-5f65172997af/serve-healthcheck-canary/0.log" Jul 08 19:54:35.106155 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:35.106064 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-rgz68_c0e7870b-512b-4301-8ebc-3479c3ac23d3/insights-operator/0.log" Jul 08 19:54:35.107670 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:35.107647 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-rgz68_c0e7870b-512b-4301-8ebc-3479c3ac23d3/insights-operator/1.log" Jul 08 19:54:35.433255 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:35.433150 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lptmt_745295a2-6e6e-4969-bbbf-ea58019b984a/kube-rbac-proxy/0.log" Jul 08 19:54:35.447208 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:35.447183 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-h89hn_0fbfa103-5cdd-4421-9061-f26746bc71a6/discovery/0.log" Jul 08 19:54:35.621591 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:35.621557 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lptmt_745295a2-6e6e-4969-bbbf-ea58019b984a/exporter/0.log" Jul 08 19:54:35.694814 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:35.694727 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lptmt_745295a2-6e6e-4969-bbbf-ea58019b984a/extractor/0.log" Jul 08 19:54:35.696153 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:35.696125 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xg7ds_1890d3fc-58bc-446b-b53f-300a7b637bab/istio-proxy/0.log" Jul 08 19:54:35.753134 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:35.753108 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xg7ds_1890d3fc-58bc-446b-b53f-300a7b637bab/istio-proxy/1.log" Jul 08 19:54:35.854976 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:35.854941 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-hf2nf_68c23503-4cdc-44d9-b1f1-60dbadd5793b/payload-pre-processing/0.log" Jul 08 19:54:35.978445 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:35.978334 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9nskb_1d6395bc-72fc-4973-a0ad-53909b703a42/payload-processing/0.log" Jul 08 19:54:38.982060 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:38.982024 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c9c9cf5d4-x8fvq_ad50df7b-244c-4f7c-9d24-94e324fa5b35/manager/0.log" Jul 08 19:54:39.487026 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:39.486989 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dc4d9889-h4phf_8c2b21b5-fba4-4572-b4f6-a900073ecdec/manager/0.log" Jul 08 19:54:39.524906 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:39.524880 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dc4d9889-h4phf_8c2b21b5-fba4-4572-b4f6-a900073ecdec/copy-manifests/0.log" Jul 08 19:54:40.360100 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:40.360065 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:40.530118 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:40.530084 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jdh6r_bdd159ef-c93c-4f21-aa89-53fab72c3541/konnectivity-agent/0.log" limit=1024 Jul 08 19:54:40.808299 ip-10-0-128-253 kubenswrapper[2563]: I0708 19:54:40.808268 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-f57846485-vscr4_38c63e5f-4aaf-41b7-afb5-99e804356d87/manager/0.log"