Oct 30 15:37:21.696451 ip-10-0-129-40 systemd[1]: Starting Kubernetes Kubelet... Oct 30 15:37:22.074641 ip-10-0-129-40 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 15:37:22.074641 ip-10-0-129-40 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 15:37:22.074641 ip-10-0-129-40 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 15:37:22.074641 ip-10-0-129-40 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 15:37:22.078338 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078256 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:37:22.078338 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078337 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:37:22.078338 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078341 2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078344 2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078347 2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078350 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078352 2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078355 2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078358 2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078360 2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078363 2572 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078365 2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078368 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078370 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078373 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078375 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078378 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078381 2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078383 2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078386 2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078388 2572 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078396 2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:37:22.078436 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078399 2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078403 2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078407 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078410 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078414 2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078418 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078421 2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078424 2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078426 2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078429 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078432 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078435 2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078438 2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078440 2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078443 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078446 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078449 2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078451 2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:37:22.078897 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078454 2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078456 2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078459 2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078461 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078464 2572 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078467 2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078469 2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078472 2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078475 2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078477 2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078479 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078482 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078484 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078487 2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078489 2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078492 2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078495 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078498 2572 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078500 2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:37:22.079331 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078503 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078505 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078508 2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078510 2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078513 2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078516 2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078518 2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078521 2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078523 2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078525 2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078528 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078530 2572 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078533 2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078537 2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078540 2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078542 2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078545 2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078547 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078550 2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078552 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:37:22.079777 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078554 2572 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078557 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078559 2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078562 2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078564 2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078566 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078569 2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078571 2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078574 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078577 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078580 2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078582 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078586 2572 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078588 2572 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078591 2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078593 2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078596 2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078599 2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078602 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078605 2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:37:22.080268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078607 2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078610 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078612 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078615 2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078617 2572 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078620 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.078623 2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079385 2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079391 2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079394 2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079396 2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079399 2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079402 2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079404 2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079407 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079410 2572 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079413 2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079415 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079418 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079420 2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:37:22.080765 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079424 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079426 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079429 2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079432 2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079434 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079437 2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079439 2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079442 2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079444 2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079447 2572 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079450 2572 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079452 2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079455 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079459 2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079463 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079466 2572 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079469 2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079472 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079474 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079477 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:37:22.081256 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079480 2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079482 2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079485 2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079487 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079490 2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079492 2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079495 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079498 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079500 2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079503 2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079505 2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079507 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079510 2572 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079513 2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079516 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079519 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079521 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079523 2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079526 2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079528 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:37:22.081782 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079531 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079534 2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079537 2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079539 2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079542 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079544 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079547 2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079549 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079551 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079554 2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079556 2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079559 2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079562 2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079564 2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079567 2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079569 2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079571 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079574 2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079576 2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:37:22.082274 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079579 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079581 2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079584 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079587 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079590 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079593 2572 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079595 2572 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079598 2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079600 2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079603 2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079605 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079608 2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079610 2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079613 2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079615 2572 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079618 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079620 2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079623 2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079625 2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:37:22.082726 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079627 2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079630 2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079632 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079634 2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079637 2572 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079640 2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079642 2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079645 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079650 2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079654 2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079656 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079659 2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079662 2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079664 2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.079667 2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080141 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080145 2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080148 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080151 2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:37:22.083208 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080154 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080157 2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080159 2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080162 2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080165 2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080168 2572 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080170 2572 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080173 2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080175 2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080178 2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080181 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080183 2572 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080186 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080188 2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080206 2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080209 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080212 2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080214 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080216 2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080219 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:37:22.084243 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080222 2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080225 2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080228 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080230 2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080233 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080235 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080238 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080240 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080243 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080246 2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080248 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080252 2572 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080254 2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080257 2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080259 2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080262 2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080264 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080267 2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080270 2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080273 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:37:22.084711 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080276 2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080278 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080281 2572 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080284 2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080286 2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080290 2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080294 2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080296 2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080299 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080302 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080304 2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080307 2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080309 2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080312 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080314 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080317 2572 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080319 2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080322 2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080324 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:37:22.085224 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080328 2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080332 2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080335 2572 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080338 2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080341 2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080344 2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080347 2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080349 2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080352 2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080354 2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080357 2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080359 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080362 2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080364 2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080367 2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080369 2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080371 2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080374 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080376 2572 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:37:22.085680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080379 2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080381 2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080383 2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080386 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080389 2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080391 2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080393 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080396 2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080398 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080403 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080405 2572 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080408 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080410 2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080412 2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080416 2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080419 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080422 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080424 2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080426 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:37:22.086142 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080429 2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080431 2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080434 2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080436 2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.080438 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080532 2572 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080539 2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080546 2572 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080550 2572 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080554 2572 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080557 2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080561 2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080565 2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080568 2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080571 2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080574 2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080578 2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080581 2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080584 2572 flags.go:64] FLAG: --cgroup-root="" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080586 2572 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080589 2572 flags.go:64] FLAG: --client-ca-file="" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080592 2572 flags.go:64] FLAG: --cloud-provider="external" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080597 2572 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080602 2572 flags.go:64] FLAG: --cluster-domain="" Oct 30 15:37:22.086633 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080605 2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080608 2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080610 2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080614 2572 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080617 2572 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080620 2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080623 2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080626 2572 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080629 2572 flags.go:64] FLAG: --contention-profiling="false" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080633 2572 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080635 2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080638 2572 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080641 2572 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080645 2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080648 2572 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080651 2572 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080653 2572 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080656 2572 flags.go:64] FLAG: --enable-server="true" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080659 2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080663 2572 flags.go:64] FLAG: --event-burst="100" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080666 2572 flags.go:64] FLAG: --event-qps="50" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080669 2572 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080672 2572 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080675 2572 flags.go:64] FLAG: --eviction-hard="" Oct 30 15:37:22.087231 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080679 2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080682 2572 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080685 2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080687 2572 flags.go:64] FLAG: --eviction-soft="" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080690 2572 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080693 2572 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080697 2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080700 2572 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080703 2572 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080706 2572 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080708 2572 flags.go:64] FLAG: --feature-gates="" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080712 2572 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080715 2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080718 2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080721 2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080724 2572 flags.go:64] FLAG: --healthz-port="10248" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080727 2572 flags.go:64] FLAG: --help="false" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080729 2572 flags.go:64] FLAG: --hostname-override="ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080733 2572 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080735 2572 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080738 2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080741 2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080744 2572 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080747 2572 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 15:37:22.088044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080750 2572 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080753 2572 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080755 2572 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080758 2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080761 2572 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080763 2572 flags.go:64] FLAG: --kube-reserved="" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080766 2572 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080769 2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080772 2572 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080775 2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080778 2572 flags.go:64] FLAG: --lock-file="" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080782 2572 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080785 2572 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080788 2572 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080794 2572 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080797 2572 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080801 2572 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080804 2572 flags.go:64] FLAG: --logging-format="text" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080806 2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080810 2572 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080812 2572 flags.go:64] FLAG: --manifest-url="" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080815 2572 flags.go:64] FLAG: --manifest-url-header="" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080820 2572 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080823 2572 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080826 2572 flags.go:64] FLAG: --max-pods="110" Oct 30 15:37:22.088820 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080829 2572 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080832 2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080835 2572 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080838 2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080841 2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080844 2572 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080846 2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080853 2572 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080856 2572 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080859 2572 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080862 2572 flags.go:64] FLAG: --pod-cidr="" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080865 2572 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080867 2572 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080870 2572 flags.go:64] FLAG: --pods-per-core="0" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080873 2572 flags.go:64] FLAG: --port="10250" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080876 2572 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080879 2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-022d59ab3c1157b41" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080882 2572 flags.go:64] FLAG: --qos-reserved="" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080886 2572 flags.go:64] FLAG: --read-only-port="10255" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080889 2572 flags.go:64] FLAG: --register-node="true" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080892 2572 flags.go:64] FLAG: --register-with-taints="" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080897 2572 flags.go:64] FLAG: --registry-burst="10" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080900 2572 flags.go:64] FLAG: --registry-qps="5" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080903 2572 flags.go:64] FLAG: --reserved-cpus="" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080906 2572 flags.go:64] FLAG: --reserved-memory="" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080910 2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 15:37:22.089461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080913 2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080916 2572 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080918 2572 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080921 2572 flags.go:64] FLAG: --runonce="false" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080924 2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080927 2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080930 2572 flags.go:64] FLAG: --seccomp-default="false" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080932 2572 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080935 2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080938 2572 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080941 2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080945 2572 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080947 2572 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080950 2572 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080953 2572 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080956 2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080959 2572 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080961 2572 flags.go:64] FLAG: --system-cgroups="" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080964 2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080969 2572 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080972 2572 flags.go:64] FLAG: --tls-cert-file="" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080975 2572 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080978 2572 flags.go:64] FLAG: --tls-min-version="" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080981 2572 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080985 2572 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 15:37:22.090096 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080988 2572 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080991 2572 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080995 2572 flags.go:64] FLAG: --v="2" Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.080999 2572 flags.go:64] FLAG: --version="false" Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.081002 2572 flags.go:64] FLAG: --vmodule="" Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.081006 2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.081010 2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081142 2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081145 2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081148 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081151 2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081153 2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081156 2572 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081158 2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081161 2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081163 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081166 2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081168 2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081170 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081173 2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081176 2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:37:22.090704 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081178 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081181 2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081183 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081186 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081188 2572 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081203 2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081206 2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081208 2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081211 2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081213 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081218 2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081220 2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081225 2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081229 2572 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081232 2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081234 2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081237 2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081239 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081242 2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081245 2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:37:22.091210 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081248 2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081250 2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081252 2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081255 2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081257 2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081260 2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081263 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081265 2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081268 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081270 2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081273 2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081275 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081278 2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081280 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081283 2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081285 2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081288 2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081290 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081293 2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081295 2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:37:22.091682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081298 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081300 2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081305 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081308 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081310 2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081314 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081316 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081319 2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081321 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081324 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081326 2572 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081330 2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081332 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081335 2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081337 2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081340 2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081342 2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081345 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081347 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:37:22.092168 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081350 2572 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081352 2572 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081354 2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081357 2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081361 2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081365 2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081368 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081371 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081374 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081376 2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081379 2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081381 2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081384 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081386 2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081389 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081392 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081395 2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081397 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081401 2572 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:37:22.092633 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081404 2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081406 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081409 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081411 2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081414 2572 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081416 2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081419 2572 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081422 2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081424 2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081427 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081429 2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081432 2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081434 2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081437 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.081462 2572 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081497 2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:37:22.093078 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081501 2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081505 2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081508 2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081510 2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081513 2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081516 2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081519 2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081521 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081523 2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081526 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081528 2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081531 2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081533 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081537 2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081540 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081543 2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081546 2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081548 2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081551 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081553 2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:37:22.093484 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081556 2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081558 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081561 2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081564 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081566 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081569 2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081572 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081575 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081577 2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081579 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081582 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081584 2572 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081587 2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081589 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081591 2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081594 2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081596 2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081599 2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081601 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:37:22.093963 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081604 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081606 2572 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081609 2572 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081611 2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081614 2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081618 2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081623 2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081626 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081630 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081632 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081635 2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081637 2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081640 2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081643 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081645 2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081648 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081650 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081653 2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081656 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:37:22.094418 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081658 2572 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081661 2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081663 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081666 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081668 2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081671 2572 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081673 2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081675 2572 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081678 2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081680 2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081683 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081686 2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081689 2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081691 2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081694 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081696 2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081699 2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081701 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081704 2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081707 2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:37:22.094855 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081710 2572 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081713 2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081716 2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081718 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081721 2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081724 2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081726 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081729 2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081732 2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081734 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081737 2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081740 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081742 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081745 2572 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081747 2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081749 2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081752 2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081754 2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081757 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081759 2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:37:22.095335 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081762 2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081766 2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081770 2572 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081772 2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081775 2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081777 2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.081780 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.090083 2572 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.090100 2572 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.090728 2572 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.090735 2572 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090799 2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090806 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090809 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090812 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090815 2572 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090817 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090820 2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090823 2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:37:22.095825 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090825 2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090828 2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090831 2572 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090834 2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090836 2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090839 2572 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090841 2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090844 2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090846 2572 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090849 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090852 2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090854 2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090857 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090859 2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090862 2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090864 2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090867 2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090869 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090871 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090874 2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:37:22.096288 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090881 2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090885 2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090887 2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090890 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090893 2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090895 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090898 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090901 2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090903 2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090905 2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090908 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090910 2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090913 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090915 2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090918 2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090920 2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090922 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090925 2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090927 2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090930 2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:37:22.096758 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090932 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090934 2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090937 2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090939 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090944 2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090948 2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090951 2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090954 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090956 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090959 2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090962 2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090964 2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090967 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090970 2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090973 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090975 2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090978 2572 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090980 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090983 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:37:22.097235 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090985 2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090988 2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090990 2572 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090993 2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090996 2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.090998 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091001 2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091003 2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091006 2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091008 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091011 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091013 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091017 2572 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091019 2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091022 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091024 2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091027 2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091029 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091031 2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:37:22.097687 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091034 2572 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091036 2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091039 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091041 2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091044 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091047 2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091049 2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091052 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091055 2572 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091057 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091060 2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091062 2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091065 2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091067 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091070 2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091072 2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091075 2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091077 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091080 2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:37:22.098136 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091083 2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.091113 2572 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091275 2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091281 2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091284 2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091287 2572 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091291 2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091294 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091297 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091300 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091304 2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091307 2572 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091311 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091314 2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091317 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091320 2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:37:22.098682 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091322 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091325 2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091328 2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091331 2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091333 2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091336 2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091338 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091341 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091343 2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091346 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091348 2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091351 2572 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091355 2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091360 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091363 2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091365 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091368 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091370 2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091373 2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091375 2572 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:37:22.099060 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091378 2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091380 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091383 2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091385 2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091388 2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091391 2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091393 2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091395 2572 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091398 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091401 2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091403 2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091405 2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091408 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091410 2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091413 2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091415 2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091418 2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091420 2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091423 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:37:22.099563 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091425 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091428 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091430 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091432 2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091435 2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091437 2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091440 2572 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091443 2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091445 2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091448 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091450 2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091453 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091455 2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091458 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091460 2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091463 2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091465 2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091467 2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091471 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:37:22.100006 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091473 2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091476 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091479 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091481 2572 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091485 2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091488 2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091491 2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091494 2572 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091497 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091499 2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091502 2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091504 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091507 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091509 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091512 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091514 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091517 2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091519 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091521 2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:37:22.100505 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091524 2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091526 2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091529 2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091531 2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091534 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091536 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091538 2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091541 2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091543 2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091546 2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091548 2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091550 2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091553 2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091555 2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091558 2572 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.091582 2572 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 15:37:22.101098 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091613 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091617 2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091620 2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091622 2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091625 2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091627 2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091630 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091632 2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091635 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091637 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091640 2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091642 2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091645 2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091648 2572 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091651 2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091653 2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091656 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091658 2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091661 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:37:22.101527 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091663 2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091666 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091668 2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091671 2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091673 2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091675 2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091678 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091680 2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091683 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091685 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091688 2572 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091693 2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091697 2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091700 2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091704 2572 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091706 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091709 2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091711 2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091714 2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:37:22.101989 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091716 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091718 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091721 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091723 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091726 2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091728 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091731 2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091733 2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091735 2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091738 2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091740 2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091742 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091745 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091747 2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091749 2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091752 2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091754 2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091757 2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091759 2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091762 2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:37:22.102588 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091764 2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091767 2572 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091769 2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091771 2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091774 2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091777 2572 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091780 2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091782 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091785 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091787 2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091790 2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091792 2572 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091795 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091797 2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091800 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091802 2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091804 2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091807 2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091810 2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091812 2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:37:22.103103 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091814 2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091817 2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091819 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091822 2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091824 2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091826 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091829 2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091834 2572 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091838 2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091842 2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091845 2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091847 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091850 2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091852 2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091855 2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091857 2572 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091859 2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091862 2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091865 2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091868 2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:37:22.103612 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091870 2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:37:22.104305 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091873 2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:37:22.104305 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091875 2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:37:22.104305 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091878 2572 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:37:22.104305 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091880 2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:37:22.104305 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091884 2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:37:22.104305 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091886 2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:37:22.104305 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:22.091889 2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:37:22.104305 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.091945 2572 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 15:37:22.104305 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.094046 2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 30 15:37:22.104305 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.094963 2572 server.go:1014] "Starting client certificate rotation" Oct 30 15:37:22.104305 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.095065 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 15:37:22.104305 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.095099 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 15:37:22.104305 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.096968 2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 15:37:22.104305 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.098362 2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 15:37:22.108305 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.108289 2572 log.go:25] "Validated CRI v1 runtime API" Oct 30 15:37:22.112113 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.112100 2572 log.go:25] "Validated CRI v1 image API" Oct 30 15:37:22.112973 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.112948 2572 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 15:37:22.118149 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.118079 2572 fs.go:135] Filesystem UUIDs: map[2656c423-a1d4-4b52-ae84-13ec73be3fb6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ac001b9c-0cdf-4e20-99dc-7e71067bfab3:/dev/nvme0n1p4] Oct 30 15:37:22.118149 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.118109 2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 30 15:37:22.125425 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.125313 2572 manager.go:217] Machine: {Timestamp:2026-10-30 15:37:22.122839979 +0000 UTC m=+0.306002206 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3092329 MemoryCapacity:32812007424 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2694473106a743752da8d61b2e0e31 SystemUUID:ec269447-3106-a743-752d-a8d61b2e0e31 BootID:bbf8ae9a-0b0e-4439-9c35-75ea962e3635 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406003712 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fc:6b:54:b1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fc:6b:54:b1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:a0:64:ac:0a:28 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812007424 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 15:37:22.125425 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.125418 2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 15:37:22.125544 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.125525 2572 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 15:37:22.127038 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.127000 2572 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 15:37:22.127188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.127034 2572 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-40.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 15:37:22.127267 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.127205 2572 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 15:37:22.127267 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.127214 2572 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 15:37:22.127267 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.127241 2572 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 15:37:22.127267 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.127254 2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 15:37:22.128248 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.128239 2572 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 15:37:22.132307 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.132295 2572 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 15:37:22.132420 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.132410 2572 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 15:37:22.133177 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.133168 2572 kubelet.go:493] "Attempting to sync node with API server" Oct 30 15:37:22.133241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.133180 2572 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 15:37:22.133241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.133206 2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 15:37:22.133241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.133216 2572 kubelet.go:405] "Adding apiserver pod source" Oct 30 15:37:22.133241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.133225 2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 15:37:22.133563 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.133548 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 15:37:22.135891 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.135876 2572 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 15:37:22.142712 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.142698 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 15:37:22.143610 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.143598 2572 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 15:37:22.143654 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.143628 2572 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 15:37:22.143811 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.143796 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 15:37:22.143869 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.143823 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 15:37:22.143869 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.143831 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 15:37:22.143869 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.143840 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 15:37:22.143869 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.143846 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 15:37:22.143869 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.143863 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 15:37:22.144617 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.144608 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 15:37:22.144647 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.144618 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 15:37:22.144647 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.144625 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 15:37:22.144647 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.144637 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 15:37:22.144724 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.144650 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 15:37:22.144724 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.144659 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 15:37:22.145461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.145451 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 15:37:22.145504 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.145463 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 15:37:22.149375 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.149362 2572 server.go:1263] "Started kubelet" Oct 30 15:37:22.149454 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.149422 2572 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 15:37:22.149513 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.149446 2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 15:37:22.149546 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.149511 2572 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 15:37:22.150154 ip-10-0-129-40 systemd[1]: Started Kubernetes Kubelet. Oct 30 15:37:22.151047 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.151031 2572 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 15:37:22.151992 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.151979 2572 server.go:317] "Adding debug handlers to kubelet server" Oct 30 15:37:22.156076 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.156052 2572 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 30 15:37:22.158873 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.158852 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 15:37:22.159502 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.159485 2572 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 15:37:22.159644 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.159607 2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found" Oct 30 15:37:22.160319 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.160303 2572 factory.go:55] Registering systemd factory Oct 30 15:37:22.160319 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.160318 2572 factory.go:223] Registration of the systemd container factory successfully Oct 30 15:37:22.160446 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.160435 2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 15:37:22.160446 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.160440 2572 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 15:37:22.160559 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.160457 2572 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 15:37:22.160559 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.160516 2572 reconstruct.go:97] "Volume reconstruction finished" Oct 30 15:37:22.160559 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.160523 2572 reconciler.go:29] "Reconciler: start to sync state" Oct 30 15:37:22.160692 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.160657 2572 factory.go:153] Registering CRI-O factory Oct 30 15:37:22.160692 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.160667 2572 factory.go:223] Registration of the crio container factory successfully Oct 30 15:37:22.160786 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.160709 2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 15:37:22.160786 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.160731 2572 factory.go:103] Registering Raw factory Oct 30 15:37:22.160786 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.160746 2572 manager.go:1199] Started watching for new ooms in manager Oct 30 15:37:22.161765 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.161752 2572 manager.go:319] Starting recovery of all containers Oct 30 15:37:22.173547 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.173526 2572 manager.go:324] Recovery completed Oct 30 15:37:22.174747 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.174732 2572 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Oct 30 15:37:22.177585 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.177571 2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 15:37:22.179448 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.179430 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory" Oct 30 15:37:22.179514 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.179468 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 15:37:22.179514 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.179484 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID" Oct 30 15:37:22.179940 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.179924 2572 cpu_manager.go:226] "Starting" policy="none" Oct 30 15:37:22.179940 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.179937 2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 15:37:22.180040 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.179939 2572 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 15:37:22.180040 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.180025 2572 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 15:37:22.180724 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.180650 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory" Oct 30 15:37:22.180809 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.180799 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 15:37:22.180844 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.180818 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID" Oct 30 15:37:22.182228 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.182216 2572 policy_none.go:50] "Start" Oct 30 15:37:22.182276 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.182231 2572 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 15:37:22.182276 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.182244 2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 15:37:22.184202 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.184180 2572 policy_none.go:44] "Start" Oct 30 15:37:22.189956 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.189929 2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-40.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 30 15:37:22.190119 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.190103 2572 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 15:37:22.191093 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.189935 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-40.ec2.internal.18e358bc3e8126df default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-40.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-40.ec2.internal,},FirstTimestamp:2026-10-30 15:37:22.149336799 +0000 UTC m=+0.332499026,LastTimestamp:2026-10-30 15:37:22.149336799 +0000 UTC m=+0.332499026,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-40.ec2.internal,}" Oct 30 15:37:22.201809 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.201653 2572 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 30 15:37:22.212071 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.212049 2572 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Oct 30 15:37:22.212071 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.212060 2572 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Oct 30 15:37:22.212071 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.212046 2572 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Oct 30 15:37:22.212342 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.212004 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-40.ec2.internal.18e358bc404ca58a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-40.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-40.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-40.ec2.internal,},FirstTimestamp:2026-10-30 15:37:22.17945025 +0000 UTC m=+0.362612484,LastTimestamp:2026-10-30 15:37:22.17945025 +0000 UTC m=+0.362612484,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-40.ec2.internal,}" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.222762 2572 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.224183 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rdcdk" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.226887 2572 manager.go:341] "Starting Device Plugin manager" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.227853 2572 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.227868 2572 server.go:87] "Starting device plugin registration server" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.228077 2572 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.228087 2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.228171 2572 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.228275 2572 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.228282 2572 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.228786 2572 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.228835 2572 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-40.ec2.internal\" not found" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.229817 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.231097 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.231114 2572 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.231129 2572 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.231157 2572 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.235363 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-40.ec2.internal.18e358bc404d05f9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-40.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-40.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-40.ec2.internal,},FirstTimestamp:2026-10-30 15:37:22.179474937 +0000 UTC m=+0.362637165,LastTimestamp:2026-10-30 15:37:22.179474937 +0000 UTC m=+0.362637165,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-40.ec2.internal,}" Oct 30 15:37:22.242722 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.240448 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rdcdk" Oct 30 15:37:22.255398 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.255339 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-40.ec2.internal.18e358bc404d40c0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-40.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-40.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-40.ec2.internal,},FirstTimestamp:2026-10-30 15:37:22.179489984 +0000 UTC m=+0.362652220,LastTimestamp:2026-10-30 15:37:22.179489984 +0000 UTC m=+0.362652220,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-40.ec2.internal,}" Oct 30 15:37:22.289700 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.289682 2572 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 15:37:22.328921 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.328883 2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 15:37:22.330087 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.330070 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory" Oct 30 15:37:22.330189 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.330100 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 15:37:22.330189 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.330123 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID" Oct 30 15:37:22.330189 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.330151 2572 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.331251 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.331230 2572 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal"] Oct 30 15:37:22.331298 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.331281 2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 15:37:22.332558 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.332543 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory" Oct 30 15:37:22.332631 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.332570 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 15:37:22.332631 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.332584 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID" Oct 30 15:37:22.334816 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.334801 2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 15:37:22.334942 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.334926 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.334989 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.334955 2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 15:37:22.335559 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.335528 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory" Oct 30 15:37:22.335559 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.335559 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 15:37:22.335673 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.335569 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID" Oct 30 15:37:22.335673 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.335539 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory" Oct 30 15:37:22.335673 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.335622 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 15:37:22.335673 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.335631 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID" Oct 30 15:37:22.337718 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.337705 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.337761 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.337729 2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 15:37:22.338420 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.338406 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory" Oct 30 15:37:22.338509 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.338430 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 15:37:22.338509 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.338439 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID" Oct 30 15:37:22.353208 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.353182 2572 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.353297 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.353220 2572 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-40.ec2.internal\": node \"ip-10-0-129-40.ec2.internal\" not found" Oct 30 15:37:22.361862 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.361843 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5ea352a0aff2f2b8b8e81204bcdb2ad9-config\") pod \"kube-apiserver-proxy-ip-10-0-129-40.ec2.internal\" (UID: \"5ea352a0aff2f2b8b8e81204bcdb2ad9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.361935 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.361867 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c39a4a8ca858e13aa8ee478d9cdc860-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"1c39a4a8ca858e13aa8ee478d9cdc860\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.361935 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.361881 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c39a4a8ca858e13aa8ee478d9cdc860-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"1c39a4a8ca858e13aa8ee478d9cdc860\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.370701 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.370668 2572 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-40.ec2.internal\" not found" node="ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.376879 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.376860 2572 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-40.ec2.internal\" not found" node="ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.395251 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.395233 2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found" Oct 30 15:37:22.462274 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.462247 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5ea352a0aff2f2b8b8e81204bcdb2ad9-config\") pod \"kube-apiserver-proxy-ip-10-0-129-40.ec2.internal\" (UID: \"5ea352a0aff2f2b8b8e81204bcdb2ad9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.462274 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.462275 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c39a4a8ca858e13aa8ee478d9cdc860-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"1c39a4a8ca858e13aa8ee478d9cdc860\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.462448 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.462290 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c39a4a8ca858e13aa8ee478d9cdc860-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"1c39a4a8ca858e13aa8ee478d9cdc860\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.462448 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.462316 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c39a4a8ca858e13aa8ee478d9cdc860-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"1c39a4a8ca858e13aa8ee478d9cdc860\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.462448 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.462338 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5ea352a0aff2f2b8b8e81204bcdb2ad9-config\") pod \"kube-apiserver-proxy-ip-10-0-129-40.ec2.internal\" (UID: \"5ea352a0aff2f2b8b8e81204bcdb2ad9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.462448 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.462354 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c39a4a8ca858e13aa8ee478d9cdc860-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"1c39a4a8ca858e13aa8ee478d9cdc860\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.495722 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.495688 2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found" Oct 30 15:37:22.595863 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.595806 2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found" Oct 30 15:37:22.675760 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.675742 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.682671 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:22.682655 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:37:22.696737 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.696722 2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found" Oct 30 15:37:22.797074 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.797054 2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found" Oct 30 15:37:22.897616 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.897563 2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found" Oct 30 15:37:22.998116 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:22.998083 2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found" Oct 30 15:37:23.094602 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.094574 2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 30 15:37:23.095049 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.094734 2572 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 15:37:23.098781 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:23.098764 2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found" Oct 30 15:37:23.199211 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:23.199174 2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found" Oct 30 15:37:23.226600 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.226578 2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 15:37:23.227765 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.227749 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory" Oct 30 15:37:23.227877 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.227781 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 15:37:23.227877 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.227796 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID" Oct 30 15:37:23.246335 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.246312 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 15:32:23 +0000 UTC" deadline="2028-07-24 08:06:25.707565016 +0000 UTC" Oct 30 15:37:23.246335 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.246333 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15184h29m2.461234091s" Oct 30 15:37:23.280589 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.280574 2572 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 15:37:23.291592 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:23.291569 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ea352a0aff2f2b8b8e81204bcdb2ad9.slice/crio-edda3baa71c3486c5355cca87b5a31e02e178daa1d6cdb2a3bb497d97e64094f WatchSource:0}: Error finding container edda3baa71c3486c5355cca87b5a31e02e178daa1d6cdb2a3bb497d97e64094f: Status 404 returned error can't find the container with id edda3baa71c3486c5355cca87b5a31e02e178daa1d6cdb2a3bb497d97e64094f Oct 30 15:37:23.293597 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.293580 2572 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 15:37:23.296316 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.296300 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 15:37:23.331608 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:23.331592 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c39a4a8ca858e13aa8ee478d9cdc860.slice/crio-c13800af7a74e91e88669cd2a1bf00e88ae1c7a7292b5e83e79ab30f3df07a41 WatchSource:0}: Error finding container c13800af7a74e91e88669cd2a1bf00e88ae1c7a7292b5e83e79ab30f3df07a41: Status 404 returned error can't find the container with id c13800af7a74e91e88669cd2a1bf00e88ae1c7a7292b5e83e79ab30f3df07a41 Oct 30 15:37:23.360704 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.360681 2572 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" Oct 30 15:37:23.383005 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.382986 2572 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 15:37:23.384047 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.384035 2572 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:37:23.407364 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.407345 2572 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 15:37:23.447472 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:23.447424 2572 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 15:37:24.014759 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.014730 2572 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 15:37:24.134253 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.134230 2572 apiserver.go:52] "Watching apiserver" Oct 30 15:37:24.160179 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.160152 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 30 15:37:24.177708 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.177680 2572 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 15:37:24.178044 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.178023 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-c4gtj","openshift-ovn-kubernetes/ovnkube-node-v2dlp","openshift-multus/multus-additional-cni-plugins-xwrzk","openshift-network-diagnostics/network-check-target-5dlc4","kube-system/global-pull-secret-syncer-kpfss","kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal","openshift-dns/node-resolver-6k2tn","openshift-image-registry/node-ca-nvxqq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal","openshift-multus/network-metrics-daemon-c6b9v","openshift-network-operator/iptables-alerter-xzc7g","kube-system/konnectivity-agent-g8vnr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4","openshift-cluster-node-tuning-operator/tuned-vnf8v"] Oct 30 15:37:24.180885 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.180855 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:37:24.183203 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.183180 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.186071 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.186050 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.189049 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.188701 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:24.189049 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.188801 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:24.190905 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.190886 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:24.191009 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.190960 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:24.193191 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.193172 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.195735 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.195661 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:37:24.197252 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.197231 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ck89s\"" Oct 30 15:37:24.197775 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.197758 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 15:37:24.197863 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.197775 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 15:37:24.198093 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.198077 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.198873 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.198851 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 15:37:24.199222 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.199167 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 15:37:24.199312 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.199228 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 15:37:24.199369 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.199343 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-khtmm\"" Oct 30 15:37:24.202795 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.202762 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.204338 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.204318 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 15:37:24.204423 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.204354 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 15:37:24.204423 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.204400 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 15:37:24.204515 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.204430 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 15:37:24.204627 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.204608 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mw526\"" Oct 30 15:37:24.204799 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.204683 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 15:37:24.205605 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.205588 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:24.205711 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.205653 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:24.208178 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.208160 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:37:24.209581 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.209559 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 15:37:24.210550 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.210518 2572 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 15:37:24.210643 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.210580 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kqb94" Oct 30 15:37:24.210744 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.210726 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:37:24.231459 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.231436 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kqb94" Oct 30 15:37:24.235776 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.235727 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" event={"ID":"5ea352a0aff2f2b8b8e81204bcdb2ad9","Type":"ContainerStarted","Data":"edda3baa71c3486c5355cca87b5a31e02e178daa1d6cdb2a3bb497d97e64094f"} Oct 30 15:37:24.236965 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.236941 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" event={"ID":"1c39a4a8ca858e13aa8ee478d9cdc860","Type":"ContainerStarted","Data":"c13800af7a74e91e88669cd2a1bf00e88ae1c7a7292b5e83e79ab30f3df07a41"} Oct 30 15:37:24.261341 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.261318 2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 15:37:24.271942 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.271893 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsp5p\" (UniqueName: \"kubernetes.io/projected/0025b75d-4f3c-4c5a-91ca-505bda44bccb-kube-api-access-hsp5p\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.271942 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.271922 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-env-overrides\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.272095 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.271940 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovnkube-script-lib\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.272095 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.271967 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn2bz\" (UniqueName: \"kubernetes.io/projected/91393a38-a119-4479-adb6-f8fe6316bce3-kube-api-access-vn2bz\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.272095 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.271995 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-kubelet\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.272095 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272092 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-systemd\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.272288 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272118 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.272288 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272150 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7df10531-4deb-4b95-ba47-9fb316775878-konnectivity-ca\") pod \"konnectivity-agent-g8vnr\" (UID: \"7df10531-4deb-4b95-ba47-9fb316775878\") " pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:37:24.272288 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272188 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-log-socket\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.272288 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272243 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.272288 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272276 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-socket-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.272484 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272305 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-registration-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.272484 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272366 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsnqq\" (UniqueName: \"kubernetes.io/projected/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-kube-api-access-vsnqq\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.272484 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272422 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30238695-c5e1-433b-b666-c822522e05e2-tmp\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.272484 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272457 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-iptables-alerter-script\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:37:24.272647 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272503 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.272647 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272541 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysctl-conf\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.272647 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272565 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-host\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.272647 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272609 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/14a2267f-e095-45cb-b0bc-99d7ab20611a-hosts-file\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:37:24.272647 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272639 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.272886 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272673 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-node-log\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.272886 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272715 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.272886 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272757 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovnkube-config\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.272886 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272789 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-kubernetes\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.272886 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272821 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-systemd\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.272886 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272845 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-var-lib-kubelet\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.272886 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272868 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-cni-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.273241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272896 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovn-node-metrics-cert\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.273241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272948 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jncfk\" (UniqueName: \"kubernetes.io/projected/bc73f5d0-671f-4cf5-8aab-325db0b149e7-kube-api-access-jncfk\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.273241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272975 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlmrz\" (UniqueName: \"kubernetes.io/projected/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-kube-api-access-zlmrz\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:37:24.273241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.272998 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-cnibin\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.273241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273018 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-cni-bin\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.273241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273041 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-cni-multus\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.273241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273067 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-etc-kubernetes\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.273241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273105 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-sys-fs\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.273241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273127 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-cnibin\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.273241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273171 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-socket-dir-parent\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.273241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273188 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysctl-d\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.273241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273205 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqmfg\" (UniqueName: \"kubernetes.io/projected/30238695-c5e1-433b-b666-c822522e05e2-kube-api-access-zqmfg\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.273241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273222 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67j8n\" (UniqueName: \"kubernetes.io/projected/3173666e-bb4d-4f89-99bd-061c19370b91-kube-api-access-67j8n\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273253 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-system-cni-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273268 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-hostroot\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273283 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-slash\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273298 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysconfig\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273316 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273367 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3173666e-bb4d-4f89-99bd-061c19370b91-host\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273388 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-conf-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273412 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-ovn\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273433 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-cni-bin\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273448 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-modprobe-d\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273463 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273554 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrwsv\" (UniqueName: \"kubernetes.io/projected/cf86f01f-c44d-46a3-be13-03c99b006285-kube-api-access-zrwsv\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273577 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-k8s-cni-cncf-io\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273597 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-etc-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273630 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-device-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.273904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273647 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-etc-selinux\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273663 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-multus-certs\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273690 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273719 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxsbg\" (UniqueName: \"kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg\") pod \"network-check-target-5dlc4\" (UID: \"c134f615-1a89-4c6a-9540-62846db76aa0\") " pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273745 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3173666e-bb4d-4f89-99bd-061c19370b91-serviceca\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273769 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-netns\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273791 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-var-lib-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273807 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-kubelet-config\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273833 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-host-slash\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273849 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7df10531-4deb-4b95-ba47-9fb316775878-agent-certs\") pod \"konnectivity-agent-g8vnr\" (UID: \"7df10531-4deb-4b95-ba47-9fb316775878\") " pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273864 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14a2267f-e095-45cb-b0bc-99d7ab20611a-tmp-dir\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273879 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvdcn\" (UniqueName: \"kubernetes.io/projected/14a2267f-e095-45cb-b0bc-99d7ab20611a-kube-api-access-lvdcn\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273895 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-system-cni-dir\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273911 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0025b75d-4f3c-4c5a-91ca-505bda44bccb-cni-binary-copy\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273926 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-kubelet\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273941 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-dbus\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:24.274726 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.273975 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-sys\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.275548 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.274000 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-lib-modules\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.275548 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.274022 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/30238695-c5e1-433b-b666-c822522e05e2-etc-tuned\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.275548 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.274038 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-systemd-units\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.275548 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.274053 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-run-netns\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.275548 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.274068 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.275548 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.274083 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-cni-netd\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.275548 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.274099 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:24.275548 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.274117 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-run\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.275548 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.274219 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-daemon-config\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.306811 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.306789 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:37:24.306915 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.306858 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 15:37:24.306915 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.306883 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 15:37:24.307028 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.306921 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 15:37:24.307184 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.307090 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 15:37:24.307184 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.307118 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ftvkm\"" Oct 30 15:37:24.307184 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.307158 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-84s7k\"" Oct 30 15:37:24.307431 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.307411 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fmlw7\"" Oct 30 15:37:24.307484 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.307441 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 15:37:24.307484 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.307452 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 15:37:24.307575 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.307499 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 15:37:24.307575 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.307548 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 15:37:24.307575 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.307565 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 15:37:24.307841 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.307815 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-xbwc7\"" Oct 30 15:37:24.307918 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.307898 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 15:37:24.307998 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.307980 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5dknm\"" Oct 30 15:37:24.308079 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.308036 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 15:37:24.308219 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.308195 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:37:24.317213 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.317181 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 15:37:24.317306 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.317231 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8vjmf\"" Oct 30 15:37:24.317306 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.317242 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 15:37:24.319838 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.319824 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 15:37:24.375191 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375162 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7df10531-4deb-4b95-ba47-9fb316775878-konnectivity-ca\") pod \"konnectivity-agent-g8vnr\" (UID: \"7df10531-4deb-4b95-ba47-9fb316775878\") " pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:37:24.375346 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375228 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-log-socket\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.375346 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375312 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.375431 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375339 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-log-socket\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.375431 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375343 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-socket-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.375431 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375392 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-registration-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.375431 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375404 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.375431 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375425 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsnqq\" (UniqueName: \"kubernetes.io/projected/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-kube-api-access-vsnqq\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.375617 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375446 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30238695-c5e1-433b-b666-c822522e05e2-tmp\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.375617 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375463 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-iptables-alerter-script\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:37:24.375617 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375488 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.375617 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375514 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysctl-conf\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.375617 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375512 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-registration-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.375617 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375542 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-socket-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.375617 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375561 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-host\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.375617 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375595 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/14a2267f-e095-45cb-b0bc-99d7ab20611a-hosts-file\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:37:24.375898 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375622 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.375898 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375649 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-node-log\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.375898 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375666 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysctl-conf\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.375898 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375674 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.375898 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375702 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovnkube-config\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.375898 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375731 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-kubernetes\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.375898 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375758 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-systemd\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.375898 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375782 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-var-lib-kubelet\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.375898 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375807 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-cni-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.375898 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375832 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovn-node-metrics-cert\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.375898 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375853 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.375898 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375859 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jncfk\" (UniqueName: \"kubernetes.io/projected/bc73f5d0-671f-4cf5-8aab-325db0b149e7-kube-api-access-jncfk\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.375898 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375875 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7df10531-4deb-4b95-ba47-9fb316775878-konnectivity-ca\") pod \"konnectivity-agent-g8vnr\" (UID: \"7df10531-4deb-4b95-ba47-9fb316775878\") " pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:37:24.375898 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375900 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlmrz\" (UniqueName: \"kubernetes.io/projected/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-kube-api-access-zlmrz\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375930 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-cnibin\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375958 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-cni-bin\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.375985 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-cni-multus\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376012 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-etc-kubernetes\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376038 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-sys-fs\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376049 2572 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376063 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-cnibin\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376088 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-socket-dir-parent\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376112 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysctl-d\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376134 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqmfg\" (UniqueName: \"kubernetes.io/projected/30238695-c5e1-433b-b666-c822522e05e2-kube-api-access-zqmfg\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376135 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-kubernetes\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376156 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67j8n\" (UniqueName: \"kubernetes.io/projected/3173666e-bb4d-4f89-99bd-061c19370b91-kube-api-access-67j8n\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376178 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-system-cni-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376178 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-cnibin\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376195 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-systemd\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376202 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-hostroot\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376247 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-host\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.376491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376274 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-node-log\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376320 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-hostroot\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376324 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376356 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysctl-d\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376369 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-cni-multus\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376371 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-var-lib-kubelet\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376402 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-etc-kubernetes\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376402 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-cni-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376427 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-slash\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376431 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/14a2267f-e095-45cb-b0bc-99d7ab20611a-hosts-file\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376437 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-cnibin\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376474 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysconfig\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376477 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-cni-bin\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376488 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-socket-dir-parent\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376512 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-system-cni-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376534 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-sys-fs\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376560 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376585 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysconfig\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.377366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376623 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3173666e-bb4d-4f89-99bd-061c19370b91-host\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376635 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-slash\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376649 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-conf-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376675 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3173666e-bb4d-4f89-99bd-061c19370b91-host\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376695 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-ovn\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376709 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-conf-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376724 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-cni-bin\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376749 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-ovn\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376751 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-modprobe-d\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376803 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-cni-bin\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376838 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376878 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrwsv\" (UniqueName: \"kubernetes.io/projected/cf86f01f-c44d-46a3-be13-03c99b006285-kube-api-access-zrwsv\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376908 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-iptables-alerter-script\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376914 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-modprobe-d\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376932 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-k8s-cni-cncf-io\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.376917 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.376969 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-k8s-cni-cncf-io\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.378206 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.377027 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs podName:cf86f01f-c44d-46a3-be13-03c99b006285 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:24.876986249 +0000 UTC m=+3.060148494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs") pod "network-metrics-daemon-c6b9v" (UID: "cf86f01f-c44d-46a3-be13-03c99b006285") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377088 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-etc-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377130 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-device-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377144 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-etc-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377161 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-etc-selinux\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377189 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovnkube-config\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377192 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-device-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377201 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-multus-certs\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377254 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377267 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-etc-selinux\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377256 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-multus-certs\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377284 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxsbg\" (UniqueName: \"kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg\") pod \"network-check-target-5dlc4\" (UID: \"c134f615-1a89-4c6a-9540-62846db76aa0\") " pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377335 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3173666e-bb4d-4f89-99bd-061c19370b91-serviceca\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377381 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-netns\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377407 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-var-lib-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377433 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-kubelet-config\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377459 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-host-slash\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:37:24.379117 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377460 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-netns\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377504 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-kubelet-config\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377597 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-var-lib-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377616 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7df10531-4deb-4b95-ba47-9fb316775878-agent-certs\") pod \"konnectivity-agent-g8vnr\" (UID: \"7df10531-4deb-4b95-ba47-9fb316775878\") " pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377646 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14a2267f-e095-45cb-b0bc-99d7ab20611a-tmp-dir\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377654 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-host-slash\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377673 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvdcn\" (UniqueName: \"kubernetes.io/projected/14a2267f-e095-45cb-b0bc-99d7ab20611a-kube-api-access-lvdcn\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377704 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-system-cni-dir\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377724 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377730 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0025b75d-4f3c-4c5a-91ca-505bda44bccb-cni-binary-copy\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377752 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-kubelet\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377773 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-dbus\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377792 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-sys\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377790 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-system-cni-dir\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377807 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-lib-modules\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377822 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/30238695-c5e1-433b-b666-c822522e05e2-etc-tuned\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377851 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3173666e-bb4d-4f89-99bd-061c19370b91-serviceca\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377862 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-systemd-units\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.379944 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377879 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-run-netns\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377895 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377913 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-cni-netd\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377918 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-dbus\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377933 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377956 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-run\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.377985 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-daemon-config\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378001 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsp5p\" (UniqueName: \"kubernetes.io/projected/0025b75d-4f3c-4c5a-91ca-505bda44bccb-kube-api-access-hsp5p\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378023 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-env-overrides\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378033 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-kubelet\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378039 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14a2267f-e095-45cb-b0bc-99d7ab20611a-tmp-dir\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378044 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovnkube-script-lib\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378082 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378112 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn2bz\" (UniqueName: \"kubernetes.io/projected/91393a38-a119-4479-adb6-f8fe6316bce3-kube-api-access-vn2bz\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378141 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-sys\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378159 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-kubelet\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378188 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-systemd\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378213 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.381050 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378248 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-lib-modules\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378322 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378495 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0025b75d-4f3c-4c5a-91ca-505bda44bccb-cni-binary-copy\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378512 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovnkube-script-lib\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378569 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-run\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378573 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-systemd\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378609 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-kubelet\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378648 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-cni-netd\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.378771 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.378823 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret podName:bbfe6e96-808d-4fb6-bce1-306f37eaf69d nodeName:}" failed. No retries permitted until 2026-10-30 15:37:24.878805894 +0000 UTC m=+3.061968120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret") pod "global-pull-secret-syncer-kpfss" (UID: "bbfe6e96-808d-4fb6-bce1-306f37eaf69d") : object "kube-system"/"original-pull-secret" not registered Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.378878 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-systemd-units\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.379022 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-run-netns\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.379705 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-daemon-config\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.379732 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.380053 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-env-overrides\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.380444 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30238695-c5e1-433b-b666-c822522e05e2-tmp\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.380624 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.382007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.381018 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/30238695-c5e1-433b-b666-c822522e05e2-etc-tuned\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.383395 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.383376 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovn-node-metrics-cert\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.383711 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.383694 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7df10531-4deb-4b95-ba47-9fb316775878-agent-certs\") pod \"konnectivity-agent-g8vnr\" (UID: \"7df10531-4deb-4b95-ba47-9fb316775878\") " pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:37:24.414369 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.414345 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67j8n\" (UniqueName: \"kubernetes.io/projected/3173666e-bb4d-4f89-99bd-061c19370b91-kube-api-access-67j8n\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:37:24.418145 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.418126 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jncfk\" (UniqueName: \"kubernetes.io/projected/bc73f5d0-671f-4cf5-8aab-325db0b149e7-kube-api-access-jncfk\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.418219 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.418126 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsnqq\" (UniqueName: \"kubernetes.io/projected/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-kube-api-access-vsnqq\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.419761 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.419701 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlmrz\" (UniqueName: \"kubernetes.io/projected/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-kube-api-access-zlmrz\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:37:24.422007 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.421984 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrwsv\" (UniqueName: \"kubernetes.io/projected/cf86f01f-c44d-46a3-be13-03c99b006285-kube-api-access-zrwsv\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:24.422493 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.422473 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqmfg\" (UniqueName: \"kubernetes.io/projected/30238695-c5e1-433b-b666-c822522e05e2-kube-api-access-zqmfg\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.423017 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.422993 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 15:37:24.423127 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.423024 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 15:37:24.423127 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.423039 2572 projected.go:196] Error preparing data for projected volume kube-api-access-gxsbg for pod openshift-network-diagnostics/network-check-target-5dlc4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 15:37:24.423127 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.423109 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg podName:c134f615-1a89-4c6a-9540-62846db76aa0 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:24.923089997 +0000 UTC m=+3.106252230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gxsbg" (UniqueName: "kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg") pod "network-check-target-5dlc4" (UID: "c134f615-1a89-4c6a-9540-62846db76aa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 15:37:24.423783 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.423755 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvdcn\" (UniqueName: \"kubernetes.io/projected/14a2267f-e095-45cb-b0bc-99d7ab20611a-kube-api-access-lvdcn\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:37:24.426307 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.426285 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsp5p\" (UniqueName: \"kubernetes.io/projected/0025b75d-4f3c-4c5a-91ca-505bda44bccb-kube-api-access-hsp5p\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.426408 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.426263 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn2bz\" (UniqueName: \"kubernetes.io/projected/91393a38-a119-4479-adb6-f8fe6316bce3-kube-api-access-vn2bz\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.494891 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.494858 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:37:24.504327 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:24.504298 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14a2267f_e095_45cb_b0bc_99d7ab20611a.slice/crio-193d22c8fdd83f86ce36aa18cf132e1cae590150b0cbbe662f18bcf9f4d4f438 WatchSource:0}: Error finding container 193d22c8fdd83f86ce36aa18cf132e1cae590150b0cbbe662f18bcf9f4d4f438: Status 404 returned error can't find the container with id 193d22c8fdd83f86ce36aa18cf132e1cae590150b0cbbe662f18bcf9f4d4f438 Oct 30 15:37:24.509304 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.509258 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:37:24.525640 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:24.525616 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1aaecc9_6c27_4adb_bf44_2c31d16fe29d.slice/crio-7100b883ded8512c8facba4245b083a52fb9801e6563e0a7958007b9904f4b5b WatchSource:0}: Error finding container 7100b883ded8512c8facba4245b083a52fb9801e6563e0a7958007b9904f4b5b: Status 404 returned error can't find the container with id 7100b883ded8512c8facba4245b083a52fb9801e6563e0a7958007b9904f4b5b Oct 30 15:37:24.527986 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.527966 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:37:24.528111 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.528092 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:37:24.529774 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.529756 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:37:24.535928 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.535905 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c4gtj" Oct 30 15:37:24.538565 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:24.538539 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30238695_c5e1_433b_b666_c822522e05e2.slice/crio-5ec9beaacfa9997f8376632ddc79f16a87902ee92f8b2c907070bd0043e839ef WatchSource:0}: Error finding container 5ec9beaacfa9997f8376632ddc79f16a87902ee92f8b2c907070bd0043e839ef: Status 404 returned error can't find the container with id 5ec9beaacfa9997f8376632ddc79f16a87902ee92f8b2c907070bd0043e839ef Oct 30 15:37:24.554969 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.554950 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:37:24.557312 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.557291 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:37:24.564279 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.564260 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:24.571094 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:24.571068 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91393a38_a119_4479_adb6_f8fe6316bce3.slice/crio-29aedf8a9de47ad4db2cef7e3c96d8a34b6e7dd5e05376a6b5c2e958063947bd WatchSource:0}: Error finding container 29aedf8a9de47ad4db2cef7e3c96d8a34b6e7dd5e05376a6b5c2e958063947bd: Status 404 returned error can't find the container with id 29aedf8a9de47ad4db2cef7e3c96d8a34b6e7dd5e05376a6b5c2e958063947bd Oct 30 15:37:24.576336 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:24.576311 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0025b75d_4f3c_4c5a_91ca_505bda44bccb.slice/crio-10f4399d852063599dc1f924ddbc866e3666ff6e11a68a84b5b0443d9e95587f WatchSource:0}: Error finding container 10f4399d852063599dc1f924ddbc866e3666ff6e11a68a84b5b0443d9e95587f: Status 404 returned error can't find the container with id 10f4399d852063599dc1f924ddbc866e3666ff6e11a68a84b5b0443d9e95587f Oct 30 15:37:24.583888 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:24.583863 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2108a090_3b2a_4b2e_a0d2_c0fe3a2ade13.slice/crio-26d5211a3f51660e7ad2eaa8a951e817188cc34d7f0f72f34b9d342b9fdca355 WatchSource:0}: Error finding container 26d5211a3f51660e7ad2eaa8a951e817188cc34d7f0f72f34b9d342b9fdca355: Status 404 returned error can't find the container with id 26d5211a3f51660e7ad2eaa8a951e817188cc34d7f0f72f34b9d342b9fdca355 Oct 30 15:37:24.584610 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:24.584588 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc73f5d0_671f_4cf5_8aab_325db0b149e7.slice/crio-adb395cb5008e7e50add661ae224bf0d113a15a629ade7d908f70c8f5118805f WatchSource:0}: Error finding container adb395cb5008e7e50add661ae224bf0d113a15a629ade7d908f70c8f5118805f: Status 404 returned error can't find the container with id adb395cb5008e7e50add661ae224bf0d113a15a629ade7d908f70c8f5118805f Oct 30 15:37:24.882795 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.882707 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:24.882795 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.882772 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:24.883020 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.882850 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 15:37:24.883020 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.882919 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs podName:cf86f01f-c44d-46a3-be13-03c99b006285 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:25.88289819 +0000 UTC m=+4.066060447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs") pod "network-metrics-daemon-c6b9v" (UID: "cf86f01f-c44d-46a3-be13-03c99b006285") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 15:37:24.883292 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.882852 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 15:37:24.883292 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.883264 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret podName:bbfe6e96-808d-4fb6-bce1-306f37eaf69d nodeName:}" failed. No retries permitted until 2026-10-30 15:37:25.883215674 +0000 UTC m=+4.066377917 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret") pod "global-pull-secret-syncer-kpfss" (UID: "bbfe6e96-808d-4fb6-bce1-306f37eaf69d") : object "kube-system"/"original-pull-secret" not registered Oct 30 15:37:24.983809 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:24.983573 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxsbg\" (UniqueName: \"kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg\") pod \"network-check-target-5dlc4\" (UID: \"c134f615-1a89-4c6a-9540-62846db76aa0\") " pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:24.983977 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.983817 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 15:37:24.983977 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.983839 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 15:37:24.983977 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.983852 2572 projected.go:196] Error preparing data for projected volume kube-api-access-gxsbg for pod openshift-network-diagnostics/network-check-target-5dlc4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 15:37:24.983977 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:24.983912 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg podName:c134f615-1a89-4c6a-9540-62846db76aa0 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:25.983893645 +0000 UTC m=+4.167055867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gxsbg" (UniqueName: "kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg") pod "network-check-target-5dlc4" (UID: "c134f615-1a89-4c6a-9540-62846db76aa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 15:37:25.231877 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.231352 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:25.231877 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:25.231527 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:25.235092 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.235019 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 15:32:25 +0000 UTC" deadline="2028-06-03 06:16:35.812268859 +0000 UTC" Oct 30 15:37:25.235092 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.235083 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13958h39m10.577189622s" Oct 30 15:37:25.260323 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.260290 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" event={"ID":"5ea352a0aff2f2b8b8e81204bcdb2ad9","Type":"ContainerStarted","Data":"f7241025f46123a9ef144f42d9036bd0b5ae64ba07e4dde2750f860a1ef0f24f"} Oct 30 15:37:25.269582 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.269550 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g8vnr" event={"ID":"7df10531-4deb-4b95-ba47-9fb316775878","Type":"ContainerStarted","Data":"b2a08d9a67abde77eea0080daa2a5bb8250c0e80fe9eeec12c657eee6d67ed78"} Oct 30 15:37:25.282943 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.282906 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xzc7g" event={"ID":"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13","Type":"ContainerStarted","Data":"26d5211a3f51660e7ad2eaa8a951e817188cc34d7f0f72f34b9d342b9fdca355"} Oct 30 15:37:25.289082 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.289030 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwrzk" event={"ID":"91393a38-a119-4479-adb6-f8fe6316bce3","Type":"ContainerStarted","Data":"29aedf8a9de47ad4db2cef7e3c96d8a34b6e7dd5e05376a6b5c2e958063947bd"} Oct 30 15:37:25.292537 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.292509 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" event={"ID":"30238695-c5e1-433b-b666-c822522e05e2","Type":"ContainerStarted","Data":"5ec9beaacfa9997f8376632ddc79f16a87902ee92f8b2c907070bd0043e839ef"} Oct 30 15:37:25.300529 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.300154 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" event={"ID":"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d","Type":"ContainerStarted","Data":"7100b883ded8512c8facba4245b083a52fb9801e6563e0a7958007b9904f4b5b"} Oct 30 15:37:25.305764 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.305729 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6k2tn" event={"ID":"14a2267f-e095-45cb-b0bc-99d7ab20611a","Type":"ContainerStarted","Data":"193d22c8fdd83f86ce36aa18cf132e1cae590150b0cbbe662f18bcf9f4d4f438"} Oct 30 15:37:25.322629 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.322598 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" event={"ID":"bc73f5d0-671f-4cf5-8aab-325db0b149e7","Type":"ContainerStarted","Data":"adb395cb5008e7e50add661ae224bf0d113a15a629ade7d908f70c8f5118805f"} Oct 30 15:37:25.331042 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.330999 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c4gtj" event={"ID":"0025b75d-4f3c-4c5a-91ca-505bda44bccb","Type":"ContainerStarted","Data":"10f4399d852063599dc1f924ddbc866e3666ff6e11a68a84b5b0443d9e95587f"} Oct 30 15:37:25.339297 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.339267 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nvxqq" event={"ID":"3173666e-bb4d-4f89-99bd-061c19370b91","Type":"ContainerStarted","Data":"63907f7263857348468b35543dbbe08956cfdd4ef04e6bbf12705dac4b29d34e"} Oct 30 15:37:25.891847 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.891809 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:25.892014 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.891891 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:25.892079 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:25.892017 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 15:37:25.892079 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:25.892077 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs podName:cf86f01f-c44d-46a3-be13-03c99b006285 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:27.892058302 +0000 UTC m=+6.075220537 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs") pod "network-metrics-daemon-c6b9v" (UID: "cf86f01f-c44d-46a3-be13-03c99b006285") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 15:37:25.892542 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:25.892518 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 15:37:25.892664 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:25.892577 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret podName:bbfe6e96-808d-4fb6-bce1-306f37eaf69d nodeName:}" failed. No retries permitted until 2026-10-30 15:37:27.892561441 +0000 UTC m=+6.075723671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret") pod "global-pull-secret-syncer-kpfss" (UID: "bbfe6e96-808d-4fb6-bce1-306f37eaf69d") : object "kube-system"/"original-pull-secret" not registered Oct 30 15:37:25.992686 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:25.992647 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxsbg\" (UniqueName: \"kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg\") pod \"network-check-target-5dlc4\" (UID: \"c134f615-1a89-4c6a-9540-62846db76aa0\") " pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:25.992867 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:25.992839 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 15:37:25.992867 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:25.992860 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 15:37:25.992979 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:25.992873 2572 projected.go:196] Error preparing data for projected volume kube-api-access-gxsbg for pod openshift-network-diagnostics/network-check-target-5dlc4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 15:37:25.992979 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:25.992931 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg podName:c134f615-1a89-4c6a-9540-62846db76aa0 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:27.992912432 +0000 UTC m=+6.176074667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gxsbg" (UniqueName: "kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg") pod "network-check-target-5dlc4" (UID: "c134f615-1a89-4c6a-9540-62846db76aa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 15:37:26.237371 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:26.234715 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:26.237371 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:26.234850 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:26.237371 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:26.235308 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:26.237371 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:26.235406 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:26.237371 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:26.235452 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 15:32:25 +0000 UTC" deadline="2028-07-14 19:29:59.284160432 +0000 UTC" Oct 30 15:37:26.237371 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:26.235474 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14955h52m33.048690569s" Oct 30 15:37:27.232029 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:27.231505 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:27.232029 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:27.231653 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:27.910600 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:27.910274 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:27.910600 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:27.910323 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:27.910600 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:27.910418 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 15:37:27.910600 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:27.910460 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret podName:bbfe6e96-808d-4fb6-bce1-306f37eaf69d nodeName:}" failed. No retries permitted until 2026-10-30 15:37:31.910447212 +0000 UTC m=+10.093609428 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret") pod "global-pull-secret-syncer-kpfss" (UID: "bbfe6e96-808d-4fb6-bce1-306f37eaf69d") : object "kube-system"/"original-pull-secret" not registered Oct 30 15:37:27.910600 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:27.910507 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 15:37:27.910600 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:27.910541 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs podName:cf86f01f-c44d-46a3-be13-03c99b006285 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:31.910529051 +0000 UTC m=+10.093691267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs") pod "network-metrics-daemon-c6b9v" (UID: "cf86f01f-c44d-46a3-be13-03c99b006285") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 15:37:28.011640 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:28.011594 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxsbg\" (UniqueName: \"kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg\") pod \"network-check-target-5dlc4\" (UID: \"c134f615-1a89-4c6a-9540-62846db76aa0\") " pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:28.011803 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:28.011779 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 15:37:28.011878 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:28.011814 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 15:37:28.011878 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:28.011827 2572 projected.go:196] Error preparing data for projected volume kube-api-access-gxsbg for pod openshift-network-diagnostics/network-check-target-5dlc4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 15:37:28.011988 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:28.011882 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg podName:c134f615-1a89-4c6a-9540-62846db76aa0 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:32.011864239 +0000 UTC m=+10.195026462 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gxsbg" (UniqueName: "kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg") pod "network-check-target-5dlc4" (UID: "c134f615-1a89-4c6a-9540-62846db76aa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 15:37:28.232469 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:28.232393 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:28.232469 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:28.232416 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:28.232689 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:28.232563 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:28.232806 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:28.232780 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:28.358055 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:28.357967 2572 generic.go:356] "Generic (PLEG): container finished" podID="91393a38-a119-4479-adb6-f8fe6316bce3" containerID="c764d98a126803368212cc09e42854f7c9a31d04e2f2c5ac89333e7fd7288384" exitCode=0 Oct 30 15:37:28.358270 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:28.358071 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwrzk" event={"ID":"91393a38-a119-4479-adb6-f8fe6316bce3","Type":"ContainerDied","Data":"c764d98a126803368212cc09e42854f7c9a31d04e2f2c5ac89333e7fd7288384"} Oct 30 15:37:28.363795 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:28.363765 2572 generic.go:356] "Generic (PLEG): container finished" podID="1c39a4a8ca858e13aa8ee478d9cdc860" containerID="e670515436d377fb4ea0675a5d6cf95d4f2273b6eabfce19630feff7389fe715" exitCode=0 Oct 30 15:37:28.363913 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:28.363817 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" event={"ID":"1c39a4a8ca858e13aa8ee478d9cdc860","Type":"ContainerDied","Data":"e670515436d377fb4ea0675a5d6cf95d4f2273b6eabfce19630feff7389fe715"} Oct 30 15:37:28.441009 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:28.440924 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" podStartSLOduration=4.440906525 podStartE2EDuration="4.440906525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:25.303718947 +0000 UTC m=+3.486881197" watchObservedRunningTime="2026-10-30 15:37:28.440906525 +0000 UTC m=+6.624068773" Oct 30 15:37:29.232325 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:29.232232 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:29.232809 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:29.232381 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:29.368563 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:29.368506 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g8vnr" event={"ID":"7df10531-4deb-4b95-ba47-9fb316775878","Type":"ContainerStarted","Data":"54134edcb3e166e4cf3edc9822c8556ecb5f189f1fc0ad2b941282b9ee5818c2"} Oct 30 15:37:29.373408 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:29.372920 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" event={"ID":"1c39a4a8ca858e13aa8ee478d9cdc860","Type":"ContainerStarted","Data":"2627bd33277369c4b1aeb1c256397639646c1d3a30cace084ee345f3191e0114"} Oct 30 15:37:29.411510 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:29.411452 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g8vnr" podStartSLOduration=2.3073588369999998 podStartE2EDuration="6.411437569s" totalImagesPullingTime="4.104078732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:29.411003758 +0000 UTC m=+7.594165997" watchObservedRunningTime="2026-10-30 15:37:29.411437569 +0000 UTC m=+7.594599808" Oct 30 15:37:30.154300 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:30.154059 2572 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:37:30.155074 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:30.154815 2572 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:37:30.198395 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:30.198348 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" podStartSLOduration=6.198328627 podStartE2EDuration="6.198328627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:29.456691995 +0000 UTC m=+7.639854238" watchObservedRunningTime="2026-10-30 15:37:30.198328627 +0000 UTC m=+8.381490869" Oct 30 15:37:30.233819 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:30.233354 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:30.233819 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:30.233496 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:30.233819 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:30.233600 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:30.233819 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:30.233713 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:30.378775 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:30.378735 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nvxqq" event={"ID":"3173666e-bb4d-4f89-99bd-061c19370b91","Type":"ContainerStarted","Data":"fac6b15fa5949324c765300381a4ea4433b8360fd59436c031ee73b36dc40b7f"} Oct 30 15:37:31.231757 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:31.231670 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:31.231940 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:31.231827 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:31.402293 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:31.401785 2572 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 15:37:31.403654 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:31.402844 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" event={"ID":"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d","Type":"ContainerStarted","Data":"a4718093e0bef88ff04db52420bceb7ddc6e5dac488f2519b8931a4a5f82f7ce"} Oct 30 15:37:31.952356 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:31.951535 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:31.952356 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:31.951594 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:31.952356 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:31.951743 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 15:37:31.952356 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:31.951799 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret podName:bbfe6e96-808d-4fb6-bce1-306f37eaf69d nodeName:}" failed. No retries permitted until 2026-10-30 15:37:39.9517822 +0000 UTC m=+18.134944430 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret") pod "global-pull-secret-syncer-kpfss" (UID: "bbfe6e96-808d-4fb6-bce1-306f37eaf69d") : object "kube-system"/"original-pull-secret" not registered Oct 30 15:37:31.952356 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:31.952227 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 15:37:31.952356 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:31.952311 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs podName:cf86f01f-c44d-46a3-be13-03c99b006285 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:39.952290399 +0000 UTC m=+18.135452614 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs") pod "network-metrics-daemon-c6b9v" (UID: "cf86f01f-c44d-46a3-be13-03c99b006285") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 15:37:32.053077 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:32.052394 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxsbg\" (UniqueName: \"kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg\") pod \"network-check-target-5dlc4\" (UID: \"c134f615-1a89-4c6a-9540-62846db76aa0\") " pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:32.053077 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:32.052636 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 15:37:32.053077 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:32.052657 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 15:37:32.053077 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:32.052671 2572 projected.go:196] Error preparing data for projected volume kube-api-access-gxsbg for pod openshift-network-diagnostics/network-check-target-5dlc4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 15:37:32.053077 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:32.052734 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg podName:c134f615-1a89-4c6a-9540-62846db76aa0 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:40.052712772 +0000 UTC m=+18.235875003 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gxsbg" (UniqueName: "kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg") pod "network-check-target-5dlc4" (UID: "c134f615-1a89-4c6a-9540-62846db76aa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 15:37:32.234003 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:32.233557 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:32.234003 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:32.233662 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:32.234003 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:32.233962 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:32.234253 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:32.234072 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:32.347960 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:32.347928 2572 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 15:37:32.408111 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:32.408068 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" event={"ID":"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d","Type":"ContainerStarted","Data":"962316ed50101d4ffbb8d907035353b17efdcf3a304f95e4612e85bdffa4d618"} Oct 30 15:37:33.232601 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:33.232065 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:33.232601 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:33.232235 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:33.235729 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:33.234969 2572 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T15:37:32.347951464Z","UUID":"81d3af3f-c58e-493c-bf17-471b2933d8ea","Handler":null,"Name":"","Endpoint":""} Oct 30 15:37:33.237138 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:33.236756 2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 15:37:33.237138 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:33.237091 2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 15:37:33.417508 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:33.417409 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwrzk" event={"ID":"91393a38-a119-4479-adb6-f8fe6316bce3","Type":"ContainerStarted","Data":"72752f4252f81eda3690f9574decf1971cf3ab4c7def1482f08aef7ee6bf70c0"} Oct 30 15:37:33.421364 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:33.421334 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6k2tn" event={"ID":"14a2267f-e095-45cb-b0bc-99d7ab20611a","Type":"ContainerStarted","Data":"9b9bdea92448e188765678117179a14976adca37173f5d9b3bb249da0597253c"} Oct 30 15:37:33.428908 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:33.428876 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c4gtj" event={"ID":"0025b75d-4f3c-4c5a-91ca-505bda44bccb","Type":"ContainerStarted","Data":"281baa2b1c241367a8af2d282669164b87ba2b960561f559be3acf281973e4a2"} Oct 30 15:37:33.470330 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:33.470255 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nvxqq" podStartSLOduration=4.881136877 podStartE2EDuration="10.470237055s" totalImagesPullingTime="5.589100178s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:30.425419769 +0000 UTC m=+8.608582020" watchObservedRunningTime="2026-10-30 15:37:33.470237055 +0000 UTC m=+11.653399294" Oct 30 15:37:33.515884 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:33.515832 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-c4gtj" podStartSLOduration=2.521550698 podStartE2EDuration="10.51581214s" totalImagesPullingTime="7.994261442s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:33.515216368 +0000 UTC m=+11.698378599" watchObservedRunningTime="2026-10-30 15:37:33.51581214 +0000 UTC m=+11.698974383" Oct 30 15:37:33.557853 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:33.557809 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-6k2tn" podStartSLOduration=2.20189523 podStartE2EDuration="10.557795633s" totalImagesPullingTime="8.355900403s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:33.557100905 +0000 UTC m=+11.740263144" watchObservedRunningTime="2026-10-30 15:37:33.557795633 +0000 UTC m=+11.740957873" Oct 30 15:37:34.231482 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:34.231450 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:34.231653 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:34.231497 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:34.231653 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:34.231618 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:34.231804 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:34.231773 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:34.433879 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:34.433839 2572 generic.go:356] "Generic (PLEG): container finished" podID="91393a38-a119-4479-adb6-f8fe6316bce3" containerID="72752f4252f81eda3690f9574decf1971cf3ab4c7def1482f08aef7ee6bf70c0" exitCode=0 Oct 30 15:37:34.434400 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:34.433939 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwrzk" event={"ID":"91393a38-a119-4479-adb6-f8fe6316bce3","Type":"ContainerDied","Data":"72752f4252f81eda3690f9574decf1971cf3ab4c7def1482f08aef7ee6bf70c0"} Oct 30 15:37:34.437012 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:34.436990 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" event={"ID":"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d","Type":"ContainerStarted","Data":"9f325489916d61f36cdc53ae1af7dd5612f8b989b7296e1d0dc819e5f579bf07"} Oct 30 15:37:34.545536 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:34.545438 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" podStartSLOduration=2.997918109 podStartE2EDuration="11.545419241s" totalImagesPullingTime="8.547501132s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:34.545345228 +0000 UTC m=+12.728507468" watchObservedRunningTime="2026-10-30 15:37:34.545419241 +0000 UTC m=+12.728581475" Oct 30 15:37:35.231889 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:35.231857 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:35.232082 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:35.232010 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:35.444278 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:35.444235 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xzc7g" event={"ID":"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13","Type":"ContainerStarted","Data":"be708029688c44899a0198159954e698570fff4273a867fb27dae24db6bcfd98"} Oct 30 15:37:36.231835 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:36.231353 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:36.231835 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:36.231495 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:36.231835 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:36.231558 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:36.231835 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:36.231657 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:36.448431 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:36.448393 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" event={"ID":"30238695-c5e1-433b-b666-c822522e05e2","Type":"ContainerStarted","Data":"4383a161ce281038bc87f9f522c68fbc1e2309fb73a2e5269fbd3db20ef39c2d"} Oct 30 15:37:36.450827 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:36.450798 2572 generic.go:356] "Generic (PLEG): container finished" podID="91393a38-a119-4479-adb6-f8fe6316bce3" containerID="01ac4a76fcc6d546ea0d8a1df7046c84e43f8528081e77a46247bd117f3d709b" exitCode=0 Oct 30 15:37:36.450940 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:36.450895 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwrzk" event={"ID":"91393a38-a119-4479-adb6-f8fe6316bce3","Type":"ContainerDied","Data":"01ac4a76fcc6d546ea0d8a1df7046c84e43f8528081e77a46247bd117f3d709b"} Oct 30 15:37:36.490134 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:36.490050 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xzc7g" podStartSLOduration=4.920403028 podStartE2EDuration="13.490035765s" totalImagesPullingTime="8.569632737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:35.487018094 +0000 UTC m=+13.670180335" watchObservedRunningTime="2026-10-30 15:37:36.490035765 +0000 UTC m=+14.673197996" Oct 30 15:37:36.490364 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:36.490333 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" podStartSLOduration=2.483945339 podStartE2EDuration="13.490325442s" totalImagesPullingTime="11.006380103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:36.489948082 +0000 UTC m=+14.673110320" watchObservedRunningTime="2026-10-30 15:37:36.490325442 +0000 UTC m=+14.673487677" Oct 30 15:37:37.232375 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:37.232166 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:37.232539 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:37.232483 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:37.455801 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:37.455725 2572 generic.go:356] "Generic (PLEG): container finished" podID="91393a38-a119-4479-adb6-f8fe6316bce3" containerID="1dd27e955f041d89def5adfde1cf27741eaf60da65862f058bb5fdc61552f2b7" exitCode=0 Oct 30 15:37:37.456388 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:37.455815 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwrzk" event={"ID":"91393a38-a119-4479-adb6-f8fe6316bce3","Type":"ContainerDied","Data":"1dd27e955f041d89def5adfde1cf27741eaf60da65862f058bb5fdc61552f2b7"} Oct 30 15:37:38.232913 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:38.232880 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:38.233102 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:38.232917 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:38.233102 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:38.233009 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:38.233471 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:38.233447 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:39.231676 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:39.231654 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:39.232017 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:39.231776 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:39.462438 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:39.462365 2572 generic.go:356] "Generic (PLEG): container finished" podID="91393a38-a119-4479-adb6-f8fe6316bce3" containerID="b25fb3029d2d3e424dd30fb55471d99545f433d39828b637ac908267eb3a3dba" exitCode=0 Oct 30 15:37:39.462438 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:39.462424 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwrzk" event={"ID":"91393a38-a119-4479-adb6-f8fe6316bce3","Type":"ContainerDied","Data":"b25fb3029d2d3e424dd30fb55471d99545f433d39828b637ac908267eb3a3dba"} Oct 30 15:37:40.018842 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:40.018818 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:40.018963 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:40.018863 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:40.018963 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:40.018955 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 15:37:40.019069 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:40.018957 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 15:37:40.019069 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:40.019004 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret podName:bbfe6e96-808d-4fb6-bce1-306f37eaf69d nodeName:}" failed. No retries permitted until 2026-10-30 15:37:56.018992956 +0000 UTC m=+34.202155184 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret") pod "global-pull-secret-syncer-kpfss" (UID: "bbfe6e96-808d-4fb6-bce1-306f37eaf69d") : object "kube-system"/"original-pull-secret" not registered Oct 30 15:37:40.019069 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:40.019019 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs podName:cf86f01f-c44d-46a3-be13-03c99b006285 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:56.01901169 +0000 UTC m=+34.202173904 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs") pod "network-metrics-daemon-c6b9v" (UID: "cf86f01f-c44d-46a3-be13-03c99b006285") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 15:37:40.119302 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:40.119280 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxsbg\" (UniqueName: \"kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg\") pod \"network-check-target-5dlc4\" (UID: \"c134f615-1a89-4c6a-9540-62846db76aa0\") " pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:40.119402 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:40.119384 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 15:37:40.119402 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:40.119399 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 15:37:40.119476 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:40.119408 2572 projected.go:196] Error preparing data for projected volume kube-api-access-gxsbg for pod openshift-network-diagnostics/network-check-target-5dlc4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 15:37:40.119476 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:40.119449 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg podName:c134f615-1a89-4c6a-9540-62846db76aa0 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:56.119434879 +0000 UTC m=+34.302597106 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gxsbg" (UniqueName: "kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg") pod "network-check-target-5dlc4" (UID: "c134f615-1a89-4c6a-9540-62846db76aa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 15:37:40.239246 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:40.239004 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:40.239571 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:40.239053 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:40.239571 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:40.239346 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:40.239571 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:40.239422 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:40.467161 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:40.467130 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" event={"ID":"bc73f5d0-671f-4cf5-8aab-325db0b149e7","Type":"ContainerStarted","Data":"ca3c11ee55b4f0c9a82e5ab3be572d16750bd0f986b78ddb12ec7a876dab022e"} Oct 30 15:37:40.467353 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:40.467222 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" event={"ID":"bc73f5d0-671f-4cf5-8aab-325db0b149e7","Type":"ContainerStarted","Data":"ea4333af7a8ec89d16eabcd74f871df44047e6031ce1a38952ad099877b11739"} Oct 30 15:37:40.467353 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:40.467273 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" event={"ID":"bc73f5d0-671f-4cf5-8aab-325db0b149e7","Type":"ContainerStarted","Data":"4ddcb7075fe8feab9fa6e8da0602245e64abd77141cb844be0e98e1f30c68aad"} Oct 30 15:37:40.467353 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:40.467307 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" event={"ID":"bc73f5d0-671f-4cf5-8aab-325db0b149e7","Type":"ContainerStarted","Data":"29fa2868b7e5d1371809b76d5833ac8d2b3f4c6bea096cde4c75129eb286e744"} Oct 30 15:37:40.467353 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:40.467349 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" event={"ID":"bc73f5d0-671f-4cf5-8aab-325db0b149e7","Type":"ContainerStarted","Data":"77e7775abd9e99aab563e97d2428f9df77b7cf03ad6bf7f24c04af88d0891198"} Oct 30 15:37:40.467515 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:40.467394 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" event={"ID":"bc73f5d0-671f-4cf5-8aab-325db0b149e7","Type":"ContainerStarted","Data":"5d0917286d3259a35596a94c2f0d30058e77453d93aa5692867b437e96b41664"} Oct 30 15:37:40.470179 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:40.470156 2572 generic.go:356] "Generic (PLEG): container finished" podID="91393a38-a119-4479-adb6-f8fe6316bce3" containerID="cb79b65197fd0ac351e62738ab215cd250806ad2fc99270111214be1d7a155a6" exitCode=0 Oct 30 15:37:40.470296 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:40.470188 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwrzk" event={"ID":"91393a38-a119-4479-adb6-f8fe6316bce3","Type":"ContainerDied","Data":"cb79b65197fd0ac351e62738ab215cd250806ad2fc99270111214be1d7a155a6"} Oct 30 15:37:41.231346 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:41.231319 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:41.231494 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:41.231412 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:41.474830 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:41.474800 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwrzk" event={"ID":"91393a38-a119-4479-adb6-f8fe6316bce3","Type":"ContainerStarted","Data":"28ddb3293e8f413995b45c06cd7f3af93d70e622df428378b434c0502a3b5cd3"} Oct 30 15:37:41.545566 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:41.545493 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xwrzk" podStartSLOduration=7.626899083 podStartE2EDuration="18.545480511s" totalImagesPullingTime="10.918581428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:41.532833253 +0000 UTC m=+19.715995500" watchObservedRunningTime="2026-10-30 15:37:41.545480511 +0000 UTC m=+19.728642754" Oct 30 15:37:42.232495 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:42.232466 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:42.232653 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:42.232539 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:42.232653 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:42.232550 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:42.232653 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:42.232629 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:42.480019 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:42.479989 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" event={"ID":"bc73f5d0-671f-4cf5-8aab-325db0b149e7","Type":"ContainerStarted","Data":"e3d421338bb74ca091ea9d8699fae760e3b5386c5eceb784b4117948b453d48c"} Oct 30 15:37:43.231426 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:43.231389 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:43.231592 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:43.231492 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:44.231892 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:44.231863 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:44.231892 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:44.231892 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:44.232422 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:44.231959 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:44.232422 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:44.232037 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:45.231415 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:45.231386 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:45.231635 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:45.231491 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:45.492967 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:45.492893 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" event={"ID":"bc73f5d0-671f-4cf5-8aab-325db0b149e7","Type":"ContainerStarted","Data":"4ed8eb366be9264f6e11d2bbdc2096c404f2831f4973e35b843a9590f736bf28"} Oct 30 15:37:45.493458 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:45.493144 2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:45.493458 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:45.493216 2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:45.509264 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:45.509097 2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:45.583561 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:45.583093 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" podStartSLOduration=7.185255514 podStartE2EDuration="22.583078606s" totalImagesPullingTime="15.397823092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:45.569953625 +0000 UTC m=+23.753115870" watchObservedRunningTime="2026-10-30 15:37:45.583078606 +0000 UTC m=+23.766240842" Oct 30 15:37:46.232249 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:46.232221 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:46.232387 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:46.232327 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:46.232488 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:46.232381 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:46.232488 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:46.232471 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:46.495441 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:46.495369 2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:46.510541 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:46.510519 2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:37:46.511170 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:46.511151 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kpfss"] Oct 30 15:37:46.511264 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:46.511253 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:46.511352 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:46.511335 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:46.514521 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:46.514497 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5dlc4"] Oct 30 15:37:46.514615 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:46.514597 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:46.514708 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:46.514691 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:46.515012 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:46.514995 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c6b9v"] Oct 30 15:37:46.515086 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:46.515076 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:46.515160 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:46.515145 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:48.065941 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:48.065909 2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:37:48.066379 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:48.066061 2572 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 15:37:48.066461 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:48.066448 2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:37:48.231611 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:48.231582 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:48.231611 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:48.231595 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:48.231611 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:48.231582 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:48.231873 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:48.231684 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:48.231873 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:48.231749 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:48.231873 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:48.231820 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:50.232317 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:50.232285 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:50.232889 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:50.232346 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:50.232889 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:50.232346 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:50.232889 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:50.232445 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dlc4" podUID="c134f615-1a89-4c6a-9540-62846db76aa0" Oct 30 15:37:50.232889 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:50.232525 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpfss" podUID="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" Oct 30 15:37:50.232889 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:50.232582 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c6b9v" podUID="cf86f01f-c44d-46a3-be13-03c99b006285" Oct 30 15:37:52.042140 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.042115 2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeReady" Oct 30 15:37:52.042449 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.042240 2572 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 30 15:37:52.096705 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.096681 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-8sbdg"] Oct 30 15:37:52.099616 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.099597 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-8sbdg" Oct 30 15:37:52.101452 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.101432 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr"] Oct 30 15:37:52.104291 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.104270 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-cnb75"] Oct 30 15:37:52.104376 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.104364 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:37:52.107376 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.107087 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xcw5d"] Oct 30 15:37:52.109057 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.109036 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-cnb75" Oct 30 15:37:52.113080 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.113060 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn"] Oct 30 15:37:52.113173 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.113132 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xcw5d" Oct 30 15:37:52.115694 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.115675 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-857f96f97f-7lt52"] Oct 30 15:37:52.115881 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.115790 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:37:52.118539 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.118522 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-tstct"] Oct 30 15:37:52.118800 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.118733 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.121823 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.121804 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-8bj4r"] Oct 30 15:37:52.121928 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.121915 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:37:52.124785 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.124728 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-s9vdp"] Oct 30 15:37:52.124878 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.124826 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.126837 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.126237 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 30 15:37:52.126837 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.126286 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 30 15:37:52.126837 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.126411 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qjh5r\"" Oct 30 15:37:52.128005 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.127987 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7ff55fd746-8jqtl"] Oct 30 15:37:52.128114 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.128088 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-s9vdp" Oct 30 15:37:52.131009 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.130992 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd"] Oct 30 15:37:52.131130 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.131095 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.133863 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.133845 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-tj9gm"] Oct 30 15:37:52.133933 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.133927 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:37:52.136649 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.136632 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-nbxc9"] Oct 30 15:37:52.136734 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.136724 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:37:52.139481 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.139465 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n"] Oct 30 15:37:52.139578 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.139563 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nbxc9" Oct 30 15:37:52.142413 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.142334 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-779688d9fc-m8g4f"] Oct 30 15:37:52.142507 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.142455 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:37:52.145433 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.145413 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bbfpq"] Oct 30 15:37:52.145575 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.145505 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.148415 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.148391 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-8sbdg"] Oct 30 15:37:52.148576 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.148505 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hrsgx"] Oct 30 15:37:52.148576 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.148509 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:52.151555 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.151534 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr"] Oct 30 15:37:52.151652 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.151572 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-cnb75"] Oct 30 15:37:52.151652 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.151593 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xcw5d"] Oct 30 15:37:52.151652 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.151639 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hrsgx" Oct 30 15:37:52.152565 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.152256 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-tstct"] Oct 30 15:37:52.167274 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.167247 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd"] Oct 30 15:37:52.168248 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.168227 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn"] Oct 30 15:37:52.169828 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.169806 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-8bj4r"] Oct 30 15:37:52.171123 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.171104 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-857f96f97f-7lt52"] Oct 30 15:37:52.174248 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.171850 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n"] Oct 30 15:37:52.174248 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.172578 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-s9vdp"] Oct 30 15:37:52.174248 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.173534 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-nbxc9"] Oct 30 15:37:52.174999 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.174979 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bbfpq"] Oct 30 15:37:52.193139 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.193118 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7ff55fd746-8jqtl"] Oct 30 15:37:52.198803 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.198778 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-tj9gm"] Oct 30 15:37:52.201693 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.200253 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-779688d9fc-m8g4f"] Oct 30 15:37:52.201693 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.200789 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hrsgx"] Oct 30 15:37:52.217315 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.217291 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:37:52.217399 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.217365 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 30 15:37:52.217452 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.217437 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 15:37:52.217505 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.217449 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 30 15:37:52.217550 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.217507 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 30 15:37:52.217607 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.217593 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:37:52.217645 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.217611 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:37:52.218503 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218488 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:37:52.218566 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218490 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 30 15:37:52.218566 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218507 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 15:37:52.218566 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218514 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 15:37:52.218566 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218535 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 30 15:37:52.218566 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218545 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:37:52.218755 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218577 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 30 15:37:52.218755 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218608 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 30 15:37:52.218755 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218647 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 15:37:52.218755 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218650 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 30 15:37:52.218755 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218654 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 15:37:52.218755 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218665 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 15:37:52.218755 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218692 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 15:37:52.218755 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218712 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 30 15:37:52.218993 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218775 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 30 15:37:52.218993 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.218810 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 15:37:52.220454 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.220384 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-ca-trust-extracted\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.220454 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.220429 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n69wr\" (UniqueName: \"kubernetes.io/projected/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-kube-api-access-n69wr\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:37:52.220576 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.220507 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-config\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:37:52.220576 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.220559 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-trusted-ca\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.220637 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.220578 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-serving-cert\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:37:52.220637 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.220622 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-installation-pull-secrets\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.220695 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.220664 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:37:52.220737 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.220691 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-trusted-ca\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.220789 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.220762 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-image-registry-private-configuration\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.220789 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.220785 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-installation-pull-secrets\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.220883 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.220859 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-image-registry-private-configuration\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.220939 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.220908 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5306d4a8-24eb-461e-9384-f71ed3b41907-trusted-ca\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:37:52.220988 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.220942 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njfdc\" (UniqueName: \"kubernetes.io/projected/eb4bb521-d604-4dbe-b464-46fb320253a5-kube-api-access-njfdc\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:37:52.221037 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.220985 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbkl9\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-kube-api-access-tbkl9\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.221037 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221012 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5306d4a8-24eb-461e-9384-f71ed3b41907-config\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:37:52.221111 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221036 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5306d4a8-24eb-461e-9384-f71ed3b41907-serving-cert\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:37:52.221111 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221078 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eb4bb521-d604-4dbe-b464-46fb320253a5-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:37:52.221111 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221100 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-registry-certificates\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.221276 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221121 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-bound-sa-token\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.221276 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221189 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eb4bb521-d604-4dbe-b464-46fb320253a5-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:37:52.221276 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221239 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b56wc\" (UniqueName: \"kubernetes.io/projected/ea29f558-cb33-4c42-9663-5db5361081b1-kube-api-access-b56wc\") pod \"volume-data-source-validator-58c9f5f46d-xcw5d\" (UID: \"ea29f558-cb33-4c42-9663-5db5361081b1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xcw5d" Oct 30 15:37:52.221276 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221267 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d95d50f-c563-4080-909a-8ec8dc3b69b7-service-ca-bundle\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.221474 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221292 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2d95d50f-c563-4080-909a-8ec8dc3b69b7-snapshots\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.221474 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221318 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:37:52.221474 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221371 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d95d50f-c563-4080-909a-8ec8dc3b69b7-tmp\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.221474 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221398 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zdpn\" (UniqueName: \"kubernetes.io/projected/2d95d50f-c563-4080-909a-8ec8dc3b69b7-kube-api-access-8zdpn\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.221474 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221422 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-certificates\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.221474 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221460 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-bound-sa-token\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.221719 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221510 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8xdt\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-kube-api-access-v8xdt\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.221719 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221556 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccndt\" (UniqueName: \"kubernetes.io/projected/5306d4a8-24eb-461e-9384-f71ed3b41907-kube-api-access-ccndt\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:37:52.221719 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221612 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d95d50f-c563-4080-909a-8ec8dc3b69b7-serving-cert\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.221719 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221638 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.221719 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221663 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjvgh\" (UniqueName: \"kubernetes.io/projected/c04fe7dc-a0c3-4626-8512-a519e8821681-kube-api-access-bjvgh\") pod \"kas-connection-checker-685895c7c9-8sbdg\" (UID: \"c04fe7dc-a0c3-4626-8512-a519e8821681\") " pod="kube-system/kas-connection-checker-685895c7c9-8sbdg" Oct 30 15:37:52.221719 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221711 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6dbr\" (UniqueName: \"kubernetes.io/projected/2d1df50d-be8d-4d79-ae09-2eb0853f313d-kube-api-access-z6dbr\") pod \"kas-connection-checker-685895c7c9-s9vdp\" (UID: \"2d1df50d-be8d-4d79-ae09-2eb0853f313d\") " pod="kube-system/kas-connection-checker-685895c7c9-s9vdp" Oct 30 15:37:52.221904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221741 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.221904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221771 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7322d7b1-561a-402f-a9bb-111264b04743-nginx-conf\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:37:52.221904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221848 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:37:52.221904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221880 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g95zs\" (UniqueName: \"kubernetes.io/projected/9eeaa1d3-abc9-4811-acad-f3a4419d967e-kube-api-access-g95zs\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:37:52.222018 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221908 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d95d50f-c563-4080-909a-8ec8dc3b69b7-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.222018 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221949 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7twh7\" (UniqueName: \"kubernetes.io/projected/eee942d8-9866-4315-ab29-d721cf68455c-kube-api-access-7twh7\") pod \"kas-connection-checker-685895c7c9-cnb75\" (UID: \"eee942d8-9866-4315-ab29-d721cf68455c\") " pod="kube-system/kas-connection-checker-685895c7c9-cnb75" Oct 30 15:37:52.222018 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.221974 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92898f00-aba8-475c-9389-1f4d3fae8103-ca-trust-extracted\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.230024 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.229997 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 15:37:52.230492 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.230475 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 15:37:52.230594 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.230520 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 30 15:37:52.230594 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.230582 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 30 15:37:52.230707 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.230605 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 30 15:37:52.230919 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.230844 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 30 15:37:52.231463 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.231448 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:52.231572 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.231453 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:52.231675 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.231657 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 30 15:37:52.231734 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.231722 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:52.241440 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.241425 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 30 15:37:52.241526 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.241442 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 30 15:37:52.241526 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.241479 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 30 15:37:52.242552 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.242536 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 15:37:52.242608 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.242567 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 30 15:37:52.242608 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.242582 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 30 15:37:52.242608 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.242592 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zt274\"" Oct 30 15:37:52.242741 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.242719 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 30 15:37:52.242741 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.242702 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 30 15:37:52.242741 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.242737 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-4vrlf\"" Oct 30 15:37:52.242848 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.242749 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 30 15:37:52.242848 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.242756 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 30 15:37:52.243163 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.243141 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-6lsxk\"" Oct 30 15:37:52.243311 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.243229 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 30 15:37:52.243562 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.243540 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 30 15:37:52.243720 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.243702 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mhkxg\"" Oct 30 15:37:52.244077 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.244057 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9c74v\"" Oct 30 15:37:52.245059 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.244986 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 15:37:52.245738 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.245721 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jlnnz\"" Oct 30 15:37:52.245940 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.245921 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 30 15:37:52.246064 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.246051 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2p9vw\"" Oct 30 15:37:52.246146 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.246130 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5wgs2\"" Oct 30 15:37:52.246290 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.246276 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vkzl2\"" Oct 30 15:37:52.246411 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.246400 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 15:37:52.246731 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.246716 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 30 15:37:52.246843 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.246828 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-zwbzw\"" Oct 30 15:37:52.246984 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.246970 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xd5fx\"" Oct 30 15:37:52.247111 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.247096 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-pxr9g\"" Oct 30 15:37:52.247629 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.247613 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 30 15:37:52.322284 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322266 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:37:52.322393 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322294 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eb4bb521-d604-4dbe-b464-46fb320253a5-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:37:52.322393 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322314 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b56wc\" (UniqueName: \"kubernetes.io/projected/ea29f558-cb33-4c42-9663-5db5361081b1-kube-api-access-b56wc\") pod \"volume-data-source-validator-58c9f5f46d-xcw5d\" (UID: \"ea29f558-cb33-4c42-9663-5db5361081b1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xcw5d" Oct 30 15:37:52.322393 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322335 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/910e7c90-9822-46c7-aa83-929c32dee95c-cert\") pod \"ingress-canary-hrsgx\" (UID: \"910e7c90-9822-46c7-aa83-929c32dee95c\") " pod="openshift-ingress-canary/ingress-canary-hrsgx" Oct 30 15:37:52.322393 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322383 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d95d50f-c563-4080-909a-8ec8dc3b69b7-service-ca-bundle\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.322597 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322407 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9dtn\" (UniqueName: \"kubernetes.io/projected/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-kube-api-access-t9dtn\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.322597 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322426 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2d95d50f-c563-4080-909a-8ec8dc3b69b7-snapshots\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.322597 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322545 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:37:52.322743 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322620 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d95d50f-c563-4080-909a-8ec8dc3b69b7-tmp\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.322743 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.322632 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 15:37:52.322743 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322654 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zdpn\" (UniqueName: \"kubernetes.io/projected/2d95d50f-c563-4080-909a-8ec8dc3b69b7-kube-api-access-8zdpn\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.322743 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.322687 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert podName:7322d7b1-561a-402f-a9bb-111264b04743 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:52.822668402 +0000 UTC m=+31.005830624 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-tstct" (UID: "7322d7b1-561a-402f-a9bb-111264b04743") : secret "networking-console-plugin-cert" not found Oct 30 15:37:52.322743 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322706 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-certificates\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.322743 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322735 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-bound-sa-token\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.323036 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322760 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8xdt\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-kube-api-access-v8xdt\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.323036 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322786 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccndt\" (UniqueName: \"kubernetes.io/projected/5306d4a8-24eb-461e-9384-f71ed3b41907-kube-api-access-ccndt\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:37:52.323036 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322814 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.323036 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322844 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d95d50f-c563-4080-909a-8ec8dc3b69b7-serving-cert\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.323036 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322861 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eb4bb521-d604-4dbe-b464-46fb320253a5-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:37:52.323036 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322867 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.323036 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322896 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjvgh\" (UniqueName: \"kubernetes.io/projected/c04fe7dc-a0c3-4626-8512-a519e8821681-kube-api-access-bjvgh\") pod \"kas-connection-checker-685895c7c9-8sbdg\" (UID: \"c04fe7dc-a0c3-4626-8512-a519e8821681\") " pod="kube-system/kas-connection-checker-685895c7c9-8sbdg" Oct 30 15:37:52.323036 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322921 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-default-certificate\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.323036 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.322983 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77495d2-eb88-48cb-b46e-510fe055471b-tmp-dir\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:52.323036 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323012 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6dbr\" (UniqueName: \"kubernetes.io/projected/2d1df50d-be8d-4d79-ae09-2eb0853f313d-kube-api-access-z6dbr\") pod \"kas-connection-checker-685895c7c9-s9vdp\" (UID: \"2d1df50d-be8d-4d79-ae09-2eb0853f313d\") " pod="kube-system/kas-connection-checker-685895c7c9-s9vdp" Oct 30 15:37:52.323036 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.323031 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 15:37:52.323036 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323037 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.323046 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-857f96f97f-7lt52: secret "image-registry-tls" not found Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323062 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl229\" (UniqueName: \"kubernetes.io/projected/910e7c90-9822-46c7-aa83-929c32dee95c-kube-api-access-jl229\") pod \"ingress-canary-hrsgx\" (UID: \"910e7c90-9822-46c7-aa83-929c32dee95c\") " pod="openshift-ingress-canary/ingress-canary-hrsgx" Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.323092 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls podName:92898f00-aba8-475c-9389-1f4d3fae8103 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:52.823074948 +0000 UTC m=+31.006237176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls") pod "image-registry-857f96f97f-7lt52" (UID: "92898f00-aba8-475c-9389-1f4d3fae8103") : secret "image-registry-tls" not found Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323092 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2d95d50f-c563-4080-909a-8ec8dc3b69b7-snapshots\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323097 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d95d50f-c563-4080-909a-8ec8dc3b69b7-service-ca-bundle\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323115 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7322d7b1-561a-402f-a9bb-111264b04743-nginx-conf\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.323216 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.323230 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ff55fd746-8jqtl: secret "image-registry-tls" not found Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.323282 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls podName:4345f2dd-427e-40d1-a5cc-43440c8cd4f0 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:52.823262364 +0000 UTC m=+31.006424584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls") pod "image-registry-7ff55fd746-8jqtl" (UID: "4345f2dd-427e-40d1-a5cc-43440c8cd4f0") : secret "image-registry-tls" not found Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323315 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323349 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-certificates\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323352 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g95zs\" (UniqueName: \"kubernetes.io/projected/9eeaa1d3-abc9-4811-acad-f3a4419d967e-kube-api-access-g95zs\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323421 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d95d50f-c563-4080-909a-8ec8dc3b69b7-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323455 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f77495d2-eb88-48cb-b46e-510fe055471b-config-volume\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323498 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7twh7\" (UniqueName: \"kubernetes.io/projected/eee942d8-9866-4315-ab29-d721cf68455c-kube-api-access-7twh7\") pod \"kas-connection-checker-685895c7c9-cnb75\" (UID: \"eee942d8-9866-4315-ab29-d721cf68455c\") " pod="kube-system/kas-connection-checker-685895c7c9-cnb75" Oct 30 15:37:52.323570 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323527 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323554 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92898f00-aba8-475c-9389-1f4d3fae8103-ca-trust-extracted\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323584 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323618 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323648 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-stats-auth\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323679 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-ca-trust-extracted\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323710 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n69wr\" (UniqueName: \"kubernetes.io/projected/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-kube-api-access-n69wr\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323739 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d95d50f-c563-4080-909a-8ec8dc3b69b7-tmp\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323751 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-config\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323782 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-trusted-ca\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323814 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7322d7b1-561a-402f-a9bb-111264b04743-nginx-conf\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.323818 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-serving-cert\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324071 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324117 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92898f00-aba8-475c-9389-1f4d3fae8103-ca-trust-extracted\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324125 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-installation-pull-secrets\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.324188 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324159 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmfvd\" (UniqueName: \"kubernetes.io/projected/f77495d2-eb88-48cb-b46e-510fe055471b-kube-api-access-kmfvd\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324216 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324529 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-config\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.324561 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324568 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d95d50f-c563-4080-909a-8ec8dc3b69b7-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324580 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-trusted-ca\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.324612 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls podName:9eeaa1d3-abc9-4811-acad-f3a4419d967e nodeName:}" failed. No retries permitted until 2026-10-30 15:37:52.82459902 +0000 UTC m=+31.007761245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-w2kcd" (UID: "9eeaa1d3-abc9-4811-acad-f3a4419d967e") : secret "samples-operator-tls" not found Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324739 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-image-registry-private-configuration\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324771 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dchj2\" (UniqueName: \"kubernetes.io/projected/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-kube-api-access-dchj2\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324800 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-installation-pull-secrets\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324830 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-image-registry-private-configuration\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324853 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5306d4a8-24eb-461e-9384-f71ed3b41907-trusted-ca\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324882 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njfdc\" (UniqueName: \"kubernetes.io/projected/eb4bb521-d604-4dbe-b464-46fb320253a5-kube-api-access-njfdc\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324907 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbkl9\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-kube-api-access-tbkl9\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324919 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-ca-trust-extracted\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.326075 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324930 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5306d4a8-24eb-461e-9384-f71ed3b41907-config\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:37:52.326775 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324951 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5306d4a8-24eb-461e-9384-f71ed3b41907-serving-cert\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:37:52.326775 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324976 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr8pl\" (UniqueName: \"kubernetes.io/projected/9c690dd2-7aa3-46c8-981b-0cc314fc2889-kube-api-access-zr8pl\") pod \"network-check-source-6f78d59f99-nbxc9\" (UID: \"9c690dd2-7aa3-46c8-981b-0cc314fc2889\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nbxc9" Oct 30 15:37:52.326775 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.324993 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-trusted-ca\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.326775 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.325006 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eb4bb521-d604-4dbe-b464-46fb320253a5-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:37:52.326775 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.325031 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-registry-certificates\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.326775 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.325062 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-bound-sa-token\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.326775 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.325684 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-registry-certificates\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.326775 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.326436 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-trusted-ca\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.326775 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.326468 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5306d4a8-24eb-461e-9384-f71ed3b41907-config\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:37:52.327180 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.326646 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-nlpvl\"" Oct 30 15:37:52.327180 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.326873 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 15:37:52.327180 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.327047 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-5ljdx\"" Oct 30 15:37:52.327180 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.327113 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qtl5k\"" Oct 30 15:37:52.327382 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.327214 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 15:37:52.328171 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.328145 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5306d4a8-24eb-461e-9384-f71ed3b41907-trusted-ca\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:37:52.328171 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.328157 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-installation-pull-secrets\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.328449 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.328405 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-serving-cert\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:37:52.328584 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.328548 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-image-registry-private-configuration\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.328584 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.328579 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-image-registry-private-configuration\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.328951 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.328902 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d95d50f-c563-4080-909a-8ec8dc3b69b7-serving-cert\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.329863 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.329844 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5306d4a8-24eb-461e-9384-f71ed3b41907-serving-cert\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:37:52.330172 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.330151 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-installation-pull-secrets\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.330246 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.330216 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eb4bb521-d604-4dbe-b464-46fb320253a5-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:37:52.350270 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.350248 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zdpn\" (UniqueName: \"kubernetes.io/projected/2d95d50f-c563-4080-909a-8ec8dc3b69b7-kube-api-access-8zdpn\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.350664 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.350641 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6dbr\" (UniqueName: \"kubernetes.io/projected/2d1df50d-be8d-4d79-ae09-2eb0853f313d-kube-api-access-z6dbr\") pod \"kas-connection-checker-685895c7c9-s9vdp\" (UID: \"2d1df50d-be8d-4d79-ae09-2eb0853f313d\") " pod="kube-system/kas-connection-checker-685895c7c9-s9vdp" Oct 30 15:37:52.350664 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.350650 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-bound-sa-token\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.350855 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.350840 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjvgh\" (UniqueName: \"kubernetes.io/projected/c04fe7dc-a0c3-4626-8512-a519e8821681-kube-api-access-bjvgh\") pod \"kas-connection-checker-685895c7c9-8sbdg\" (UID: \"c04fe7dc-a0c3-4626-8512-a519e8821681\") " pod="kube-system/kas-connection-checker-685895c7c9-8sbdg" Oct 30 15:37:52.350890 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.350859 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8xdt\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-kube-api-access-v8xdt\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.350966 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.350945 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccndt\" (UniqueName: \"kubernetes.io/projected/5306d4a8-24eb-461e-9384-f71ed3b41907-kube-api-access-ccndt\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:37:52.351039 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.351026 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b56wc\" (UniqueName: \"kubernetes.io/projected/ea29f558-cb33-4c42-9663-5db5361081b1-kube-api-access-b56wc\") pod \"volume-data-source-validator-58c9f5f46d-xcw5d\" (UID: \"ea29f558-cb33-4c42-9663-5db5361081b1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xcw5d" Oct 30 15:37:52.365172 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.365154 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbkl9\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-kube-api-access-tbkl9\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.365629 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.365612 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-bound-sa-token\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.365903 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.365878 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njfdc\" (UniqueName: \"kubernetes.io/projected/eb4bb521-d604-4dbe-b464-46fb320253a5-kube-api-access-njfdc\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:37:52.366001 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.365943 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g95zs\" (UniqueName: \"kubernetes.io/projected/9eeaa1d3-abc9-4811-acad-f3a4419d967e-kube-api-access-g95zs\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:37:52.366069 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.366018 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7twh7\" (UniqueName: \"kubernetes.io/projected/eee942d8-9866-4315-ab29-d721cf68455c-kube-api-access-7twh7\") pod \"kas-connection-checker-685895c7c9-cnb75\" (UID: \"eee942d8-9866-4315-ab29-d721cf68455c\") " pod="kube-system/kas-connection-checker-685895c7c9-cnb75" Oct 30 15:37:52.366069 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.366057 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n69wr\" (UniqueName: \"kubernetes.io/projected/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-kube-api-access-n69wr\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:37:52.414662 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.414642 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-8sbdg" Oct 30 15:37:52.421125 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.421105 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:37:52.426444 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.426426 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr8pl\" (UniqueName: \"kubernetes.io/projected/9c690dd2-7aa3-46c8-981b-0cc314fc2889-kube-api-access-zr8pl\") pod \"network-check-source-6f78d59f99-nbxc9\" (UID: \"9c690dd2-7aa3-46c8-981b-0cc314fc2889\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nbxc9" Oct 30 15:37:52.426501 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.426476 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:37:52.426539 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.426511 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/910e7c90-9822-46c7-aa83-929c32dee95c-cert\") pod \"ingress-canary-hrsgx\" (UID: \"910e7c90-9822-46c7-aa83-929c32dee95c\") " pod="openshift-ingress-canary/ingress-canary-hrsgx" Oct 30 15:37:52.426576 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.426543 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9dtn\" (UniqueName: \"kubernetes.io/projected/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-kube-api-access-t9dtn\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.426625 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.426591 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.426673 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.426631 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-default-certificate\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.426673 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.426661 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77495d2-eb88-48cb-b46e-510fe055471b-tmp-dir\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:52.426749 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.426709 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl229\" (UniqueName: \"kubernetes.io/projected/910e7c90-9822-46c7-aa83-929c32dee95c-kube-api-access-jl229\") pod \"ingress-canary-hrsgx\" (UID: \"910e7c90-9822-46c7-aa83-929c32dee95c\") " pod="openshift-ingress-canary/ingress-canary-hrsgx" Oct 30 15:37:52.426749 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.426728 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 15:37:52.426839 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.426749 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f77495d2-eb88-48cb-b46e-510fe055471b-config-volume\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:52.426839 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.426789 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs podName:0a1bfd21-ddfa-4820-8356-49214c4cfc3b nodeName:}" failed. No retries permitted until 2026-10-30 15:37:52.926765496 +0000 UTC m=+31.109927726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs") pod "router-default-779688d9fc-m8g4f" (UID: "0a1bfd21-ddfa-4820-8356-49214c4cfc3b") : secret "router-metrics-certs-default" not found Oct 30 15:37:52.426839 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.426827 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:52.426977 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.426865 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.426977 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.426896 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:37:52.426977 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.426933 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-stats-auth\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.426977 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.426955 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 15:37:52.426977 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.426973 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmfvd\" (UniqueName: \"kubernetes.io/projected/f77495d2-eb88-48cb-b46e-510fe055471b-kube-api-access-kmfvd\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:52.427232 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.427008 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls podName:f77495d2-eb88-48cb-b46e-510fe055471b nodeName:}" failed. No retries permitted until 2026-10-30 15:37:52.92699646 +0000 UTC m=+31.110158674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls") pod "dns-default-bbfpq" (UID: "f77495d2-eb88-48cb-b46e-510fe055471b") : secret "dns-default-metrics-tls" not found Oct 30 15:37:52.427232 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.427054 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dchj2\" (UniqueName: \"kubernetes.io/projected/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-kube-api-access-dchj2\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:37:52.427343 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.427298 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f77495d2-eb88-48cb-b46e-510fe055471b-config-volume\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:52.427343 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.427316 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:37:52.427437 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.427412 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle podName:0a1bfd21-ddfa-4820-8356-49214c4cfc3b nodeName:}" failed. No retries permitted until 2026-10-30 15:37:52.927396552 +0000 UTC m=+31.110558783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle") pod "router-default-779688d9fc-m8g4f" (UID: "0a1bfd21-ddfa-4820-8356-49214c4cfc3b") : configmap references non-existent config key: service-ca.crt Oct 30 15:37:52.427487 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.427469 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 15:37:52.427534 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.427516 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls podName:e54831c9-77f6-4bce-b8fc-bbfd1838dddd nodeName:}" failed. No retries permitted until 2026-10-30 15:37:52.927497331 +0000 UTC m=+31.110659546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-xqb8n" (UID: "e54831c9-77f6-4bce-b8fc-bbfd1838dddd") : secret "cluster-monitoring-operator-tls" not found Oct 30 15:37:52.427662 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.427639 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77495d2-eb88-48cb-b46e-510fe055471b-tmp-dir\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:52.429317 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.429298 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-cnb75" Oct 30 15:37:52.429729 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.429713 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/910e7c90-9822-46c7-aa83-929c32dee95c-cert\") pod \"ingress-canary-hrsgx\" (UID: \"910e7c90-9822-46c7-aa83-929c32dee95c\") " pod="openshift-ingress-canary/ingress-canary-hrsgx" Oct 30 15:37:52.429849 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.429824 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-default-certificate\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.429992 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.429973 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-stats-auth\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.435122 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.435106 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xcw5d" Oct 30 15:37:52.441675 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.441657 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:37:52.453410 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.453385 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9dtn\" (UniqueName: \"kubernetes.io/projected/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-kube-api-access-t9dtn\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.455366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.455343 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr8pl\" (UniqueName: \"kubernetes.io/projected/9c690dd2-7aa3-46c8-981b-0cc314fc2889-kube-api-access-zr8pl\") pod \"network-check-source-6f78d59f99-nbxc9\" (UID: \"9c690dd2-7aa3-46c8-981b-0cc314fc2889\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nbxc9" Oct 30 15:37:52.458122 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.458094 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl229\" (UniqueName: \"kubernetes.io/projected/910e7c90-9822-46c7-aa83-929c32dee95c-kube-api-access-jl229\") pod \"ingress-canary-hrsgx\" (UID: \"910e7c90-9822-46c7-aa83-929c32dee95c\") " pod="openshift-ingress-canary/ingress-canary-hrsgx" Oct 30 15:37:52.460136 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.460049 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dchj2\" (UniqueName: \"kubernetes.io/projected/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-kube-api-access-dchj2\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:37:52.460136 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.460054 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmfvd\" (UniqueName: \"kubernetes.io/projected/f77495d2-eb88-48cb-b46e-510fe055471b-kube-api-access-kmfvd\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:52.471134 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.470957 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:37:52.486777 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.486601 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-s9vdp" Oct 30 15:37:52.498671 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.498484 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:37:52.517031 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.510504 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nbxc9" Oct 30 15:37:52.531760 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.531670 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hrsgx" Oct 30 15:37:52.751492 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.751467 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-8sbdg"] Oct 30 15:37:52.752893 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:52.752857 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc04fe7dc_a0c3_4626_8512_a519e8821681.slice/crio-801d2f35ec081124f933cca6d52aab3efebc43063b61af2c3ceb3f61e47000cb WatchSource:0}: Error finding container 801d2f35ec081124f933cca6d52aab3efebc43063b61af2c3ceb3f61e47000cb: Status 404 returned error can't find the container with id 801d2f35ec081124f933cca6d52aab3efebc43063b61af2c3ceb3f61e47000cb Oct 30 15:37:52.767420 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.767386 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xcw5d"] Oct 30 15:37:52.774503 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.772314 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-cnb75"] Oct 30 15:37:52.774503 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.774045 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr"] Oct 30 15:37:52.775268 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:52.774781 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4bb521_d604_4dbe_b464_46fb320253a5.slice/crio-de891953be9ade7bbb17c6b8e7cee8871cb4b970879f0a38f6acd4642a740442 WatchSource:0}: Error finding container de891953be9ade7bbb17c6b8e7cee8871cb4b970879f0a38f6acd4642a740442: Status 404 returned error can't find the container with id de891953be9ade7bbb17c6b8e7cee8871cb4b970879f0a38f6acd4642a740442 Oct 30 15:37:52.779784 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.779767 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn"] Oct 30 15:37:52.782797 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:52.782774 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23eff76f_8eed_4d55_88b9_1ae63cb1e9ec.slice/crio-a83821ca18e0e2b64d3caa415f922b5b97cfc1c78eaf1787ff1b0e4699b130d6 WatchSource:0}: Error finding container a83821ca18e0e2b64d3caa415f922b5b97cfc1c78eaf1787ff1b0e4699b130d6: Status 404 returned error can't find the container with id a83821ca18e0e2b64d3caa415f922b5b97cfc1c78eaf1787ff1b0e4699b130d6 Oct 30 15:37:52.831795 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.831770 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:37:52.831920 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.831819 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:52.831920 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.831840 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:52.831920 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.831919 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 15:37:52.832090 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.831927 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 15:37:52.832090 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.831977 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:37:52.832090 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.831995 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert podName:7322d7b1-561a-402f-a9bb-111264b04743 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:53.831977974 +0000 UTC m=+32.015140208 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-tstct" (UID: "7322d7b1-561a-402f-a9bb-111264b04743") : secret "networking-console-plugin-cert" not found Oct 30 15:37:52.832090 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.832045 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 15:37:52.832090 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.832048 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 15:37:52.832090 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.832076 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-857f96f97f-7lt52: secret "image-registry-tls" not found Oct 30 15:37:52.832090 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.832080 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls podName:9eeaa1d3-abc9-4811-acad-f3a4419d967e nodeName:}" failed. No retries permitted until 2026-10-30 15:37:53.832069782 +0000 UTC m=+32.015232005 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-w2kcd" (UID: "9eeaa1d3-abc9-4811-acad-f3a4419d967e") : secret "samples-operator-tls" not found Oct 30 15:37:52.832090 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.831929 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ff55fd746-8jqtl: secret "image-registry-tls" not found Oct 30 15:37:52.832545 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.832128 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls podName:92898f00-aba8-475c-9389-1f4d3fae8103 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:53.832110852 +0000 UTC m=+32.015273083 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls") pod "image-registry-857f96f97f-7lt52" (UID: "92898f00-aba8-475c-9389-1f4d3fae8103") : secret "image-registry-tls" not found Oct 30 15:37:52.832545 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.832148 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls podName:4345f2dd-427e-40d1-a5cc-43440c8cd4f0 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:53.832138213 +0000 UTC m=+32.015300429 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls") pod "image-registry-7ff55fd746-8jqtl" (UID: "4345f2dd-427e-40d1-a5cc-43440c8cd4f0") : secret "image-registry-tls" not found Oct 30 15:37:52.865009 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.864887 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-8bj4r"] Oct 30 15:37:52.866637 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:52.866613 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d95d50f_c563_4080_909a_8ec8dc3b69b7.slice/crio-7d429a794704486ee79bdbc437afe85d138f2bffdcd0d2cc6eed39c04d03250c WatchSource:0}: Error finding container 7d429a794704486ee79bdbc437afe85d138f2bffdcd0d2cc6eed39c04d03250c: Status 404 returned error can't find the container with id 7d429a794704486ee79bdbc437afe85d138f2bffdcd0d2cc6eed39c04d03250c Oct 30 15:37:52.870717 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.870699 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-s9vdp"] Oct 30 15:37:52.871854 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.871839 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-tj9gm"] Oct 30 15:37:52.875444 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:52.875420 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d1df50d_be8d_4d79_ae09_2eb0853f313d.slice/crio-85a01e79e10328907cc525f8fb341eefe4898bdd62ff4a885933fcd376ee6fc7 WatchSource:0}: Error finding container 85a01e79e10328907cc525f8fb341eefe4898bdd62ff4a885933fcd376ee6fc7: Status 404 returned error can't find the container with id 85a01e79e10328907cc525f8fb341eefe4898bdd62ff4a885933fcd376ee6fc7 Oct 30 15:37:52.875751 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:52.875715 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5306d4a8_24eb_461e_9384_f71ed3b41907.slice/crio-f8f31e26c79f285a251c441f8180d9c32b4f29e9ee9f1e9509da00d9a6f36be1 WatchSource:0}: Error finding container f8f31e26c79f285a251c441f8180d9c32b4f29e9ee9f1e9509da00d9a6f36be1: Status 404 returned error can't find the container with id f8f31e26c79f285a251c441f8180d9c32b4f29e9ee9f1e9509da00d9a6f36be1 Oct 30 15:37:52.899164 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.899104 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-nbxc9"] Oct 30 15:37:52.910683 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.910462 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hrsgx"] Oct 30 15:37:52.912680 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:52.912626 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod910e7c90_9822_46c7_aa83_929c32dee95c.slice/crio-178c5d7f1cec252740eeb224e34e1b09e1f276bd51b71559cc18b9910a859e83 WatchSource:0}: Error finding container 178c5d7f1cec252740eeb224e34e1b09e1f276bd51b71559cc18b9910a859e83: Status 404 returned error can't find the container with id 178c5d7f1cec252740eeb224e34e1b09e1f276bd51b71559cc18b9910a859e83 Oct 30 15:37:52.932639 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.932613 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.932737 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.932711 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:52.932790 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.932740 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:52.932790 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.932755 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 15:37:52.932790 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:52.932769 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:37:52.932906 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.932806 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs podName:0a1bfd21-ddfa-4820-8356-49214c4cfc3b nodeName:}" failed. No retries permitted until 2026-10-30 15:37:53.932789239 +0000 UTC m=+32.115951474 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs") pod "router-default-779688d9fc-m8g4f" (UID: "0a1bfd21-ddfa-4820-8356-49214c4cfc3b") : secret "router-metrics-certs-default" not found Oct 30 15:37:52.932906 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.932839 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 15:37:52.932906 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.932903 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls podName:e54831c9-77f6-4bce-b8fc-bbfd1838dddd nodeName:}" failed. No retries permitted until 2026-10-30 15:37:53.932892329 +0000 UTC m=+32.116054545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-xqb8n" (UID: "e54831c9-77f6-4bce-b8fc-bbfd1838dddd") : secret "cluster-monitoring-operator-tls" not found Oct 30 15:37:52.933007 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.932907 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 15:37:52.933007 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.932936 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls podName:f77495d2-eb88-48cb-b46e-510fe055471b nodeName:}" failed. No retries permitted until 2026-10-30 15:37:53.932927189 +0000 UTC m=+32.116089411 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls") pod "dns-default-bbfpq" (UID: "f77495d2-eb88-48cb-b46e-510fe055471b") : secret "dns-default-metrics-tls" not found Oct 30 15:37:52.933007 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:52.932972 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle podName:0a1bfd21-ddfa-4820-8356-49214c4cfc3b nodeName:}" failed. No retries permitted until 2026-10-30 15:37:53.932962471 +0000 UTC m=+32.116124699 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle") pod "router-default-779688d9fc-m8g4f" (UID: "0a1bfd21-ddfa-4820-8356-49214c4cfc3b") : configmap references non-existent config key: service-ca.crt Oct 30 15:37:53.514473 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.514351 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" event={"ID":"5306d4a8-24eb-461e-9384-f71ed3b41907","Type":"ContainerStarted","Data":"f8f31e26c79f285a251c441f8180d9c32b4f29e9ee9f1e9509da00d9a6f36be1"} Oct 30 15:37:53.517000 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.516929 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" event={"ID":"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec","Type":"ContainerStarted","Data":"a83821ca18e0e2b64d3caa415f922b5b97cfc1c78eaf1787ff1b0e4699b130d6"} Oct 30 15:37:53.520167 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.519370 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-cnb75" event={"ID":"eee942d8-9866-4315-ab29-d721cf68455c","Type":"ContainerStarted","Data":"5656efab4f81db0865e4c27922940f81f089b488bc15379e5717d00cbf00bdc6"} Oct 30 15:37:53.520167 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.519410 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-cnb75" event={"ID":"eee942d8-9866-4315-ab29-d721cf68455c","Type":"ContainerStarted","Data":"eedd95ddefbd506f780979180a6db801a452bca507e96ddffe75b29b555f15c1"} Oct 30 15:37:53.525908 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.525825 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" event={"ID":"eb4bb521-d604-4dbe-b464-46fb320253a5","Type":"ContainerStarted","Data":"de891953be9ade7bbb17c6b8e7cee8871cb4b970879f0a38f6acd4642a740442"} Oct 30 15:37:53.527310 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.527286 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nbxc9" event={"ID":"9c690dd2-7aa3-46c8-981b-0cc314fc2889","Type":"ContainerStarted","Data":"b9b4add9e6248070ae9e2de7d0e34209fef2c40ab9812dd1e22ea756052e6699"} Oct 30 15:37:53.531561 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.531067 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-8sbdg" event={"ID":"c04fe7dc-a0c3-4626-8512-a519e8821681","Type":"ContainerStarted","Data":"7d7338f3fb7b407c8857b5beef061cf2376d22aab4be9a893adb06f42a722e12"} Oct 30 15:37:53.531561 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.531105 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-8sbdg" event={"ID":"c04fe7dc-a0c3-4626-8512-a519e8821681","Type":"ContainerStarted","Data":"801d2f35ec081124f933cca6d52aab3efebc43063b61af2c3ceb3f61e47000cb"} Oct 30 15:37:53.534529 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.534063 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-s9vdp" event={"ID":"2d1df50d-be8d-4d79-ae09-2eb0853f313d","Type":"ContainerStarted","Data":"e2e2377d65b3e8ccc65bedbb918869289ec6a4329b018db45f3dca5e1adc10b3"} Oct 30 15:37:53.534529 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.534102 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-s9vdp" event={"ID":"2d1df50d-be8d-4d79-ae09-2eb0853f313d","Type":"ContainerStarted","Data":"85a01e79e10328907cc525f8fb341eefe4898bdd62ff4a885933fcd376ee6fc7"} Oct 30 15:37:53.536866 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.536843 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8bj4r" event={"ID":"2d95d50f-c563-4080-909a-8ec8dc3b69b7","Type":"ContainerStarted","Data":"7d429a794704486ee79bdbc437afe85d138f2bffdcd0d2cc6eed39c04d03250c"} Oct 30 15:37:53.542979 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.542920 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xcw5d" event={"ID":"ea29f558-cb33-4c42-9663-5db5361081b1","Type":"ContainerStarted","Data":"5e17aba9bf8cf3d0b3ce4706701e0c0bcddf8f4a9092b910c642eca0dc7e705e"} Oct 30 15:37:53.546251 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.546182 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hrsgx" event={"ID":"910e7c90-9822-46c7-aa83-929c32dee95c","Type":"ContainerStarted","Data":"178c5d7f1cec252740eeb224e34e1b09e1f276bd51b71559cc18b9910a859e83"} Oct 30 15:37:53.567930 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.566945 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-cnb75" podStartSLOduration=374.566931974 podStartE2EDuration="6m14.566931974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:31:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:53.563884048 +0000 UTC m=+31.747046325" watchObservedRunningTime="2026-10-30 15:37:53.566931974 +0000 UTC m=+31.750094210" Oct 30 15:37:53.654955 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.654897 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-8sbdg" podStartSLOduration=374.654880221 podStartE2EDuration="6m14.654880221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:31:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:53.608621707 +0000 UTC m=+31.791783961" watchObservedRunningTime="2026-10-30 15:37:53.654880221 +0000 UTC m=+31.838042458" Oct 30 15:37:53.655164 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.654996 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-s9vdp" podStartSLOduration=374.65498902 podStartE2EDuration="6m14.65498902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:31:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:53.654432645 +0000 UTC m=+31.837594905" watchObservedRunningTime="2026-10-30 15:37:53.65498902 +0000 UTC m=+31.838151257" Oct 30 15:37:53.849878 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.849790 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:53.850014 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.849954 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:37:53.850114 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.850078 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:37:53.850173 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.850157 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:53.850352 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.850335 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 15:37:53.850398 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.850358 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-857f96f97f-7lt52: secret "image-registry-tls" not found Oct 30 15:37:53.850439 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.850428 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls podName:92898f00-aba8-475c-9389-1f4d3fae8103 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:55.850406758 +0000 UTC m=+34.033568996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls") pod "image-registry-857f96f97f-7lt52" (UID: "92898f00-aba8-475c-9389-1f4d3fae8103") : secret "image-registry-tls" not found Oct 30 15:37:53.850892 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.850869 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 15:37:53.850944 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.850896 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ff55fd746-8jqtl: secret "image-registry-tls" not found Oct 30 15:37:53.850973 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.850946 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls podName:4345f2dd-427e-40d1-a5cc-43440c8cd4f0 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:55.85093 +0000 UTC m=+34.034092225 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls") pod "image-registry-7ff55fd746-8jqtl" (UID: "4345f2dd-427e-40d1-a5cc-43440c8cd4f0") : secret "image-registry-tls" not found Oct 30 15:37:53.851032 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.851018 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 15:37:53.851072 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.851062 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls podName:9eeaa1d3-abc9-4811-acad-f3a4419d967e nodeName:}" failed. No retries permitted until 2026-10-30 15:37:55.851050445 +0000 UTC m=+34.034212675 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-w2kcd" (UID: "9eeaa1d3-abc9-4811-acad-f3a4419d967e") : secret "samples-operator-tls" not found Oct 30 15:37:53.851137 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.851123 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 15:37:53.851169 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.851165 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert podName:7322d7b1-561a-402f-a9bb-111264b04743 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:55.851153654 +0000 UTC m=+34.034315884 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-tstct" (UID: "7322d7b1-561a-402f-a9bb-111264b04743") : secret "networking-console-plugin-cert" not found Oct 30 15:37:53.950590 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.950560 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:53.950590 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.950600 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:53.950904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.950629 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:37:53.950904 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:53.950738 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:53.950904 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.950862 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 15:37:53.951127 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.950923 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs podName:0a1bfd21-ddfa-4820-8356-49214c4cfc3b nodeName:}" failed. No retries permitted until 2026-10-30 15:37:55.950904678 +0000 UTC m=+34.134066908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs") pod "router-default-779688d9fc-m8g4f" (UID: "0a1bfd21-ddfa-4820-8356-49214c4cfc3b") : secret "router-metrics-certs-default" not found Oct 30 15:37:53.951379 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.951363 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle podName:0a1bfd21-ddfa-4820-8356-49214c4cfc3b nodeName:}" failed. No retries permitted until 2026-10-30 15:37:55.951350765 +0000 UTC m=+34.134512979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle") pod "router-default-779688d9fc-m8g4f" (UID: "0a1bfd21-ddfa-4820-8356-49214c4cfc3b") : configmap references non-existent config key: service-ca.crt Oct 30 15:37:53.951483 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.951376 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 15:37:53.951483 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.951438 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 15:37:53.951483 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.951441 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls podName:e54831c9-77f6-4bce-b8fc-bbfd1838dddd nodeName:}" failed. No retries permitted until 2026-10-30 15:37:55.951423207 +0000 UTC m=+34.134585445 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-xqb8n" (UID: "e54831c9-77f6-4bce-b8fc-bbfd1838dddd") : secret "cluster-monitoring-operator-tls" not found Oct 30 15:37:53.951483 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:53.951486 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls podName:f77495d2-eb88-48cb-b46e-510fe055471b nodeName:}" failed. No retries permitted until 2026-10-30 15:37:55.951474785 +0000 UTC m=+34.134637001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls") pod "dns-default-bbfpq" (UID: "f77495d2-eb88-48cb-b46e-510fe055471b") : secret "dns-default-metrics-tls" not found Oct 30 15:37:54.226353 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:54.226319 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:37:54.465942 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:54.465869 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:37:55.569473 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:55.568740 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xcw5d" event={"ID":"ea29f558-cb33-4c42-9663-5db5361081b1","Type":"ContainerStarted","Data":"4a55756d87d684272eef02bab19485df353efd314700a11fabbcb43f50f88de5"} Oct 30 15:37:55.610804 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:55.610752 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xcw5d" podStartSLOduration=318.223471817 podStartE2EDuration="5m20.61073392s" totalImagesPullingTime="2.387262103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:55.609592517 +0000 UTC m=+33.792754752" watchObservedRunningTime="2026-10-30 15:37:55.61073392 +0000 UTC m=+33.793896155" Oct 30 15:37:55.881928 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:55.881859 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:37:55.882124 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:55.881963 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:37:55.882124 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:55.882014 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:55.882124 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.882040 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 15:37:55.882124 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.882119 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls podName:9eeaa1d3-abc9-4811-acad-f3a4419d967e nodeName:}" failed. No retries permitted until 2026-10-30 15:37:59.882098866 +0000 UTC m=+38.065261095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-w2kcd" (UID: "9eeaa1d3-abc9-4811-acad-f3a4419d967e") : secret "samples-operator-tls" not found Oct 30 15:37:55.882369 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.882141 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 15:37:55.882369 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.882154 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ff55fd746-8jqtl: secret "image-registry-tls" not found Oct 30 15:37:55.882369 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.882219 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls podName:4345f2dd-427e-40d1-a5cc-43440c8cd4f0 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:59.882186732 +0000 UTC m=+38.065348951 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls") pod "image-registry-7ff55fd746-8jqtl" (UID: "4345f2dd-427e-40d1-a5cc-43440c8cd4f0") : secret "image-registry-tls" not found Oct 30 15:37:55.882369 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:55.882048 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:55.882369 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.882286 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 15:37:55.882369 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.882306 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 15:37:55.882369 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.882320 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-857f96f97f-7lt52: secret "image-registry-tls" not found Oct 30 15:37:55.882369 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.882323 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert podName:7322d7b1-561a-402f-a9bb-111264b04743 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:59.882311693 +0000 UTC m=+38.065473914 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-tstct" (UID: "7322d7b1-561a-402f-a9bb-111264b04743") : secret "networking-console-plugin-cert" not found Oct 30 15:37:55.882931 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.882411 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls podName:92898f00-aba8-475c-9389-1f4d3fae8103 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:59.882398911 +0000 UTC m=+38.065561126 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls") pod "image-registry-857f96f97f-7lt52" (UID: "92898f00-aba8-475c-9389-1f4d3fae8103") : secret "image-registry-tls" not found Oct 30 15:37:55.890615 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:55.890512 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:37:55.983544 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:55.983497 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:55.983706 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:55.983621 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:37:55.983706 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:55.983646 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:37:55.983706 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:55.983673 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:37:55.983917 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.983819 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 15:37:55.983917 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.983874 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls podName:e54831c9-77f6-4bce-b8fc-bbfd1838dddd nodeName:}" failed. No retries permitted until 2026-10-30 15:37:59.983856875 +0000 UTC m=+38.167019101 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-xqb8n" (UID: "e54831c9-77f6-4bce-b8fc-bbfd1838dddd") : secret "cluster-monitoring-operator-tls" not found Oct 30 15:37:55.984310 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.984280 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 15:37:55.984451 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.984339 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs podName:0a1bfd21-ddfa-4820-8356-49214c4cfc3b nodeName:}" failed. No retries permitted until 2026-10-30 15:37:59.984323774 +0000 UTC m=+38.167485989 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs") pod "router-default-779688d9fc-m8g4f" (UID: "0a1bfd21-ddfa-4820-8356-49214c4cfc3b") : secret "router-metrics-certs-default" not found Oct 30 15:37:55.984451 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.984414 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 15:37:55.984567 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.984459 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls podName:f77495d2-eb88-48cb-b46e-510fe055471b nodeName:}" failed. No retries permitted until 2026-10-30 15:37:59.984438417 +0000 UTC m=+38.167600634 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls") pod "dns-default-bbfpq" (UID: "f77495d2-eb88-48cb-b46e-510fe055471b") : secret "dns-default-metrics-tls" not found Oct 30 15:37:55.984567 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:55.984530 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle podName:0a1bfd21-ddfa-4820-8356-49214c4cfc3b nodeName:}" failed. No retries permitted until 2026-10-30 15:37:59.9845188 +0000 UTC m=+38.167681014 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle") pod "router-default-779688d9fc-m8g4f" (UID: "0a1bfd21-ddfa-4820-8356-49214c4cfc3b") : configmap references non-existent config key: service-ca.crt Oct 30 15:37:56.074928 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:56.074888 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:37:56.084619 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:56.084597 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:56.084737 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:56.084663 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:37:56.084804 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:56.084765 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 30 15:37:56.084861 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:56.084808 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs podName:cf86f01f-c44d-46a3-be13-03c99b006285 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:28.084794316 +0000 UTC m=+66.267956542 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs") pod "network-metrics-daemon-c6b9v" (UID: "cf86f01f-c44d-46a3-be13-03c99b006285") : secret "metrics-daemon-secret" not found Oct 30 15:37:56.091316 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:56.091277 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:56.149021 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:56.148911 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:37:56.185906 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:56.185404 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxsbg\" (UniqueName: \"kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg\") pod \"network-check-target-5dlc4\" (UID: \"c134f615-1a89-4c6a-9540-62846db76aa0\") " pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:56.189137 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:56.189111 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxsbg\" (UniqueName: \"kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg\") pod \"network-check-target-5dlc4\" (UID: \"c134f615-1a89-4c6a-9540-62846db76aa0\") " pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:56.454186 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:56.454073 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:37:56.482147 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:56.482116 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kpfss"] Oct 30 15:37:56.491146 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:56.491116 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbfe6e96_808d_4fb6_bce1_306f37eaf69d.slice/crio-5f96514100a9ea2337b5b1724185f3e4afa30bca37a086217079d652803e0403 WatchSource:0}: Error finding container 5f96514100a9ea2337b5b1724185f3e4afa30bca37a086217079d652803e0403: Status 404 returned error can't find the container with id 5f96514100a9ea2337b5b1724185f3e4afa30bca37a086217079d652803e0403 Oct 30 15:37:56.574903 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:56.574876 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kpfss" event={"ID":"bbfe6e96-808d-4fb6-bce1-306f37eaf69d","Type":"ContainerStarted","Data":"5f96514100a9ea2337b5b1724185f3e4afa30bca37a086217079d652803e0403"} Oct 30 15:37:56.813577 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:56.813449 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5dlc4"] Oct 30 15:37:56.856517 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:37:56.856471 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc134f615_1a89_4c6a_9540_62846db76aa0.slice/crio-639b913c660246019f02aba1039b0e529e9615ee8d2d6190a6c690eec0078d97 WatchSource:0}: Error finding container 639b913c660246019f02aba1039b0e529e9615ee8d2d6190a6c690eec0078d97: Status 404 returned error can't find the container with id 639b913c660246019f02aba1039b0e529e9615ee8d2d6190a6c690eec0078d97 Oct 30 15:37:57.579309 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:57.579231 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5dlc4" event={"ID":"c134f615-1a89-4c6a-9540-62846db76aa0","Type":"ContainerStarted","Data":"639b913c660246019f02aba1039b0e529e9615ee8d2d6190a6c690eec0078d97"} Oct 30 15:37:57.581251 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:57.581226 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/0.log" Oct 30 15:37:57.581368 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:57.581265 2572 generic.go:356] "Generic (PLEG): container finished" podID="5306d4a8-24eb-461e-9384-f71ed3b41907" containerID="1641983a234eb83528a801a2bb285f32839a6c1ac50d6915ab7284662dd8faa9" exitCode=255 Oct 30 15:37:57.581368 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:57.581313 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" event={"ID":"5306d4a8-24eb-461e-9384-f71ed3b41907","Type":"ContainerDied","Data":"1641983a234eb83528a801a2bb285f32839a6c1ac50d6915ab7284662dd8faa9"} Oct 30 15:37:57.581620 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:57.581604 2572 scope.go:122] "RemoveContainer" containerID="1641983a234eb83528a801a2bb285f32839a6c1ac50d6915ab7284662dd8faa9" Oct 30 15:37:58.591716 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.591682 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" event={"ID":"eb4bb521-d604-4dbe-b464-46fb320253a5","Type":"ContainerStarted","Data":"872551740a91bb5a5bc3769875a9fda151fdc88d9421d63d36fcc19f6348b772"} Oct 30 15:37:58.598679 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.598620 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nbxc9" event={"ID":"9c690dd2-7aa3-46c8-981b-0cc314fc2889","Type":"ContainerStarted","Data":"554f80dac5a8bdc68bdd38f48a51632b0ea66937915172fc9255709a22ddbd24"} Oct 30 15:37:58.601366 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.601344 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8bj4r" event={"ID":"2d95d50f-c563-4080-909a-8ec8dc3b69b7","Type":"ContainerStarted","Data":"5d317a217cff3b1c3acf014be915a9f3d56ce1429225596913cf33f050eaa0f0"} Oct 30 15:37:58.603864 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.603557 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hrsgx" event={"ID":"910e7c90-9822-46c7-aa83-929c32dee95c","Type":"ContainerStarted","Data":"562f6a6548d472b2c24c2d8296fd1e55a0bc76bd946a12d136029dd3fbb3c47b"} Oct 30 15:37:58.606752 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.606735 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/1.log" Oct 30 15:37:58.608255 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.607541 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/0.log" Oct 30 15:37:58.608255 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.607574 2572 generic.go:356] "Generic (PLEG): container finished" podID="5306d4a8-24eb-461e-9384-f71ed3b41907" containerID="f0bff60ba47b0b67c8bd6bf33c99bf64c08bc4973fb67c080a9b011c64cfbfd2" exitCode=255 Oct 30 15:37:58.608255 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.607637 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" event={"ID":"5306d4a8-24eb-461e-9384-f71ed3b41907","Type":"ContainerDied","Data":"f0bff60ba47b0b67c8bd6bf33c99bf64c08bc4973fb67c080a9b011c64cfbfd2"} Oct 30 15:37:58.608255 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.607677 2572 scope.go:122] "RemoveContainer" containerID="1641983a234eb83528a801a2bb285f32839a6c1ac50d6915ab7284662dd8faa9" Oct 30 15:37:58.608255 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.608013 2572 scope.go:122] "RemoveContainer" containerID="f0bff60ba47b0b67c8bd6bf33c99bf64c08bc4973fb67c080a9b011c64cfbfd2" Oct 30 15:37:58.608255 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:58.608184 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tj9gm_openshift-console-operator(5306d4a8-24eb-461e-9384-f71ed3b41907)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" podUID="5306d4a8-24eb-461e-9384-f71ed3b41907" Oct 30 15:37:58.635620 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.635555 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" podStartSLOduration=319.209948657 podStartE2EDuration="5m24.635536628s" totalImagesPullingTime="5.425587971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:58.633452883 +0000 UTC m=+36.816615120" watchObservedRunningTime="2026-10-30 15:37:58.635536628 +0000 UTC m=+36.818698864" Oct 30 15:37:58.640851 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.639897 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" event={"ID":"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec","Type":"ContainerStarted","Data":"26783cef90471bbc7b1711c864ef301d264d485c12d4485b627b82496daaf929"} Oct 30 15:37:58.725740 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.723995 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nbxc9" podStartSLOduration=324.157614145 podStartE2EDuration="5m29.723976996s" totalImagesPullingTime="5.566362851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:58.723276105 +0000 UTC m=+36.906438343" watchObservedRunningTime="2026-10-30 15:37:58.723976996 +0000 UTC m=+36.907139233" Oct 30 15:37:58.809912 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.809801 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hrsgx" podStartSLOduration=1.508506052 podStartE2EDuration="6.809782055s" totalImagesPullingTime="5.301276003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:37:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:58.766740837 +0000 UTC m=+36.949903077" watchObservedRunningTime="2026-10-30 15:37:58.809782055 +0000 UTC m=+36.992944293" Oct 30 15:37:58.810133 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.810103 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-8bj4r" podStartSLOduration=316.477926058 podStartE2EDuration="5m21.810091905s" totalImagesPullingTime="5.332165847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:32:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:58.808267044 +0000 UTC m=+36.991429297" watchObservedRunningTime="2026-10-30 15:37:58.810091905 +0000 UTC m=+36.993254143" Oct 30 15:37:58.848208 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:58.848134 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" podStartSLOduration=319.439403402 podStartE2EDuration="5m24.848115618s" totalImagesPullingTime="5.408712216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:58.847823265 +0000 UTC m=+37.030985528" watchObservedRunningTime="2026-10-30 15:37:58.848115618 +0000 UTC m=+37.031277858" Oct 30 15:37:59.203837 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.203794 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:37:59.371761 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.371727 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:37:59.485965 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.485890 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-hgpjh"] Oct 30 15:37:59.511947 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.511105 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:37:59.512615 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.512557 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-hgpjh"] Oct 30 15:37:59.526463 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.526445 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 30 15:37:59.539529 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.538571 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 30 15:37:59.539529 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.538677 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-grmxw\"" Oct 30 15:37:59.539529 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.538766 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 30 15:37:59.539529 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.538879 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 30 15:37:59.619225 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.619175 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-signing-cabundle\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:37:59.619642 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.619258 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-signing-key\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:37:59.619642 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.619411 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxv8s\" (UniqueName: \"kubernetes.io/projected/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-kube-api-access-jxv8s\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:37:59.647567 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.647545 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/1.log" Oct 30 15:37:59.648287 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.648270 2572 scope.go:122] "RemoveContainer" containerID="f0bff60ba47b0b67c8bd6bf33c99bf64c08bc4973fb67c080a9b011c64cfbfd2" Oct 30 15:37:59.648455 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:59.648436 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tj9gm_openshift-console-operator(5306d4a8-24eb-461e-9384-f71ed3b41907)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" podUID="5306d4a8-24eb-461e-9384-f71ed3b41907" Oct 30 15:37:59.720675 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.720638 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxv8s\" (UniqueName: \"kubernetes.io/projected/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-kube-api-access-jxv8s\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:37:59.720807 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.720757 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-signing-cabundle\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:37:59.720867 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.720805 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-signing-key\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:37:59.722082 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.721855 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-signing-cabundle\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:37:59.737693 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.737629 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-signing-key\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:37:59.747595 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.747564 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxv8s\" (UniqueName: \"kubernetes.io/projected/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-kube-api-access-jxv8s\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:37:59.840543 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.837287 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:37:59.840543 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.839533 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-jzd98"] Oct 30 15:37:59.841678 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.841654 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:37:59.856927 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.856903 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-fv7tg"] Oct 30 15:37:59.857491 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.857451 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-jzd98" Oct 30 15:37:59.874700 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.874379 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 30 15:37:59.874700 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.874566 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vr6wd\"" Oct 30 15:37:59.874700 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.874670 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 30 15:37:59.889344 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.888053 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-jzd98"] Oct 30 15:37:59.889344 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.888141 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fv7tg" Oct 30 15:37:59.890814 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.890742 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-fv7tg"] Oct 30 15:37:59.925655 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.925625 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72qnj\" (UniqueName: \"kubernetes.io/projected/50c89e19-a6a1-4abe-906d-e2548c00e23a-kube-api-access-72qnj\") pod \"migrator-66df5d496b-jzd98\" (UID: \"50c89e19-a6a1-4abe-906d-e2548c00e23a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-jzd98" Oct 30 15:37:59.925741 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.925681 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:37:59.925800 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.925766 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:37:59.925841 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.925826 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:37:59.925875 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.925861 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:37:59.925924 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:37:59.925910 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snbvb\" (UniqueName: \"kubernetes.io/projected/521c6b52-045a-447e-9d9e-49c0de21c0a2-kube-api-access-snbvb\") pod \"migrator-66df5d496b-fv7tg\" (UID: \"521c6b52-045a-447e-9d9e-49c0de21c0a2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fv7tg" Oct 30 15:37:59.926136 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:59.926122 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 15:37:59.926206 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:59.926179 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls podName:9eeaa1d3-abc9-4811-acad-f3a4419d967e nodeName:}" failed. No retries permitted until 2026-10-30 15:38:07.926161261 +0000 UTC m=+46.109323481 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-w2kcd" (UID: "9eeaa1d3-abc9-4811-acad-f3a4419d967e") : secret "samples-operator-tls" not found Oct 30 15:37:59.938537 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:59.938423 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 15:37:59.938537 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:59.938438 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 15:37:59.938537 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:59.938445 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-857f96f97f-7lt52: secret "image-registry-tls" not found Oct 30 15:37:59.938537 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:59.938500 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls podName:92898f00-aba8-475c-9389-1f4d3fae8103 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:07.938483374 +0000 UTC m=+46.121645601 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls") pod "image-registry-857f96f97f-7lt52" (UID: "92898f00-aba8-475c-9389-1f4d3fae8103") : secret "image-registry-tls" not found Oct 30 15:37:59.938537 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:59.938513 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert podName:7322d7b1-561a-402f-a9bb-111264b04743 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:07.938507464 +0000 UTC m=+46.121669678 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-tstct" (UID: "7322d7b1-561a-402f-a9bb-111264b04743") : secret "networking-console-plugin-cert" not found Oct 30 15:37:59.938756 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:59.938549 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 15:37:59.938756 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:59.938559 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ff55fd746-8jqtl: secret "image-registry-tls" not found Oct 30 15:37:59.938756 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:37:59.938607 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls podName:4345f2dd-427e-40d1-a5cc-43440c8cd4f0 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:07.938580766 +0000 UTC m=+46.121743002 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls") pod "image-registry-7ff55fd746-8jqtl" (UID: "4345f2dd-427e-40d1-a5cc-43440c8cd4f0") : secret "image-registry-tls" not found Oct 30 15:38:00.011606 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.011517 2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g47gl"] Oct 30 15:38:00.020480 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.020449 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:00.026714 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.026685 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:38:00.026874 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.026732 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72qnj\" (UniqueName: \"kubernetes.io/projected/50c89e19-a6a1-4abe-906d-e2548c00e23a-kube-api-access-72qnj\") pod \"migrator-66df5d496b-jzd98\" (UID: \"50c89e19-a6a1-4abe-906d-e2548c00e23a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-jzd98" Oct 30 15:38:00.026988 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.026970 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 15:38:00.027090 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.027077 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls podName:e54831c9-77f6-4bce-b8fc-bbfd1838dddd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:08.027056841 +0000 UTC m=+46.210219077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-xqb8n" (UID: "e54831c9-77f6-4bce-b8fc-bbfd1838dddd") : secret "cluster-monitoring-operator-tls" not found Oct 30 15:38:00.027525 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.027500 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:00.027638 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.027560 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:38:00.027638 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.027585 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snbvb\" (UniqueName: \"kubernetes.io/projected/521c6b52-045a-447e-9d9e-49c0de21c0a2-kube-api-access-snbvb\") pod \"migrator-66df5d496b-fv7tg\" (UID: \"521c6b52-045a-447e-9d9e-49c0de21c0a2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fv7tg" Oct 30 15:38:00.027638 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.027613 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:00.027757 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.027634 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 15:38:00.027757 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.027692 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs podName:0a1bfd21-ddfa-4820-8356-49214c4cfc3b nodeName:}" failed. No retries permitted until 2026-10-30 15:38:08.027673633 +0000 UTC m=+46.210835861 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs") pod "router-default-779688d9fc-m8g4f" (UID: "0a1bfd21-ddfa-4820-8356-49214c4cfc3b") : secret "router-metrics-certs-default" not found Oct 30 15:38:00.027757 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.027711 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 15:38:00.027757 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.027755 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls podName:f77495d2-eb88-48cb-b46e-510fe055471b nodeName:}" failed. No retries permitted until 2026-10-30 15:38:08.02774181 +0000 UTC m=+46.210904027 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls") pod "dns-default-bbfpq" (UID: "f77495d2-eb88-48cb-b46e-510fe055471b") : secret "dns-default-metrics-tls" not found Oct 30 15:38:00.027900 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.027798 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle podName:0a1bfd21-ddfa-4820-8356-49214c4cfc3b nodeName:}" failed. No retries permitted until 2026-10-30 15:38:08.027779645 +0000 UTC m=+46.210941859 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle") pod "router-default-779688d9fc-m8g4f" (UID: "0a1bfd21-ddfa-4820-8356-49214c4cfc3b") : configmap references non-existent config key: service-ca.crt Oct 30 15:38:00.033432 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.033405 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.045704 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.045675 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g47gl"] Oct 30 15:38:00.059084 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.059043 2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 15:38:00.061303 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.061268 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rf9nq\"" Oct 30 15:38:00.062212 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.062096 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72qnj\" (UniqueName: \"kubernetes.io/projected/50c89e19-a6a1-4abe-906d-e2548c00e23a-kube-api-access-72qnj\") pod \"migrator-66df5d496b-jzd98\" (UID: \"50c89e19-a6a1-4abe-906d-e2548c00e23a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-jzd98" Oct 30 15:38:00.064492 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.064458 2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 15:38:00.065171 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.065142 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snbvb\" (UniqueName: \"kubernetes.io/projected/521c6b52-045a-447e-9d9e-49c0de21c0a2-kube-api-access-snbvb\") pod \"migrator-66df5d496b-fv7tg\" (UID: \"521c6b52-045a-447e-9d9e-49c0de21c0a2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fv7tg" Oct 30 15:38:00.128527 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.128364 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.128527 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.128448 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj24q\" (UniqueName: \"kubernetes.io/projected/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-api-access-dj24q\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.128747 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.128602 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9bcd436-d777-4ca7-a840-bce674d247fd-data-volume\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.128747 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.128632 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.128747 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.128734 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b9bcd436-d777-4ca7-a840-bce674d247fd-crio-socket\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.172957 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.172925 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-hgpjh"] Oct 30 15:38:00.174216 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:38:00.174175 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cd4f53f_cc43_43da_bde1_c9e8d5a23e6e.slice/crio-28b815f8ae9435007e7952a029539da96090d444abfe27739689310371565426 WatchSource:0}: Error finding container 28b815f8ae9435007e7952a029539da96090d444abfe27739689310371565426: Status 404 returned error can't find the container with id 28b815f8ae9435007e7952a029539da96090d444abfe27739689310371565426 Oct 30 15:38:00.221938 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.221914 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-jzd98" Oct 30 15:38:00.230266 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.230248 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9bcd436-d777-4ca7-a840-bce674d247fd-data-volume\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.230355 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.230275 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.230355 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.230315 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b9bcd436-d777-4ca7-a840-bce674d247fd-crio-socket\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.230457 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.230412 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.230457 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.230432 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 15:38:00.230457 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.230453 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj24q\" (UniqueName: \"kubernetes.io/projected/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-api-access-dj24q\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.230602 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.230496 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls podName:b9bcd436-d777-4ca7-a840-bce674d247fd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:00.730480952 +0000 UTC m=+38.913643185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g47gl" (UID: "b9bcd436-d777-4ca7-a840-bce674d247fd") : secret "insights-runtime-extractor-tls" not found Oct 30 15:38:00.230602 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.230521 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b9bcd436-d777-4ca7-a840-bce674d247fd-crio-socket\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.230602 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.230584 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9bcd436-d777-4ca7-a840-bce674d247fd-data-volume\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.230747 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.230623 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 15:38:00.230747 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.230670 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm podName:b9bcd436-d777-4ca7-a840-bce674d247fd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:00.730658136 +0000 UTC m=+38.913820351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g47gl" (UID: "b9bcd436-d777-4ca7-a840-bce674d247fd") : configmap "kube-rbac-proxy" not found Oct 30 15:38:00.242354 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.242338 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fv7tg" Oct 30 15:38:00.261431 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.261404 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:00.268452 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.268426 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj24q\" (UniqueName: \"kubernetes.io/projected/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-api-access-dj24q\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.553068 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.553039 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-jzd98"] Oct 30 15:38:00.553767 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:38:00.553742 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50c89e19_a6a1_4abe_906d_e2548c00e23a.slice/crio-3c3fff7263bb1bbea6d51328fa0da0d1ebce890bc0664b6d54c459b03df4ce54 WatchSource:0}: Error finding container 3c3fff7263bb1bbea6d51328fa0da0d1ebce890bc0664b6d54c459b03df4ce54: Status 404 returned error can't find the container with id 3c3fff7263bb1bbea6d51328fa0da0d1ebce890bc0664b6d54c459b03df4ce54 Oct 30 15:38:00.582206 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.582174 2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-fv7tg"] Oct 30 15:38:00.582869 ip-10-0-129-40 kubenswrapper[2572]: W1030 15:38:00.582847 2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod521c6b52_045a_447e_9d9e_49c0de21c0a2.slice/crio-4beedffb5268777c1fc11108dd5832c033e8373f74c9c8f010e445cf7bc86669 WatchSource:0}: Error finding container 4beedffb5268777c1fc11108dd5832c033e8373f74c9c8f010e445cf7bc86669: Status 404 returned error can't find the container with id 4beedffb5268777c1fc11108dd5832c033e8373f74c9c8f010e445cf7bc86669 Oct 30 15:38:00.653740 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.653700 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5dlc4" event={"ID":"c134f615-1a89-4c6a-9540-62846db76aa0","Type":"ContainerStarted","Data":"af355400172e8521582cb92cb78b44ce108ef48feff5a41d9b9b602595462a9c"} Oct 30 15:38:00.654130 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.653872 2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:38:00.655325 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.655299 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" event={"ID":"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e","Type":"ContainerStarted","Data":"e71c8f7f7822644e6c46b2fa534943e7dda7782869c48435a41d6dd6b6791179"} Oct 30 15:38:00.655441 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.655342 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" event={"ID":"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e","Type":"ContainerStarted","Data":"28b815f8ae9435007e7952a029539da96090d444abfe27739689310371565426"} Oct 30 15:38:00.656717 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.656689 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fv7tg" event={"ID":"521c6b52-045a-447e-9d9e-49c0de21c0a2","Type":"ContainerStarted","Data":"4beedffb5268777c1fc11108dd5832c033e8373f74c9c8f010e445cf7bc86669"} Oct 30 15:38:00.657960 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.657938 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-jzd98" event={"ID":"50c89e19-a6a1-4abe-906d-e2548c00e23a","Type":"ContainerStarted","Data":"3c3fff7263bb1bbea6d51328fa0da0d1ebce890bc0664b6d54c459b03df4ce54"} Oct 30 15:38:00.714973 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.714931 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5dlc4" podStartSLOduration=34.103626399 podStartE2EDuration="37.714917712s" totalImagesPullingTime="3.611291313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:00.713106386 +0000 UTC m=+38.896268623" watchObservedRunningTime="2026-10-30 15:38:00.714917712 +0000 UTC m=+38.898079949" Oct 30 15:38:00.736884 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.736856 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.737241 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.737217 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:00.737460 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.737443 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 15:38:00.737537 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.737459 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 15:38:00.737537 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.737511 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm podName:b9bcd436-d777-4ca7-a840-bce674d247fd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:01.73749616 +0000 UTC m=+39.920658380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g47gl" (UID: "b9bcd436-d777-4ca7-a840-bce674d247fd") : configmap "kube-rbac-proxy" not found Oct 30 15:38:00.737537 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:00.737526 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls podName:b9bcd436-d777-4ca7-a840-bce674d247fd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:01.737518016 +0000 UTC m=+39.920680233 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g47gl" (UID: "b9bcd436-d777-4ca7-a840-bce674d247fd") : secret "insights-runtime-extractor-tls" not found Oct 30 15:38:00.760453 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.760406 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" podStartSLOduration=1.760383022 podStartE2EDuration="1.760383022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:37:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:00.759228523 +0000 UTC m=+38.942390777" watchObservedRunningTime="2026-10-30 15:38:00.760383022 +0000 UTC m=+38.943545259" Oct 30 15:38:00.951212 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:00.951166 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:01.208434 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:01.208348 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:01.391969 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:01.391846 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:01.587298 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:01.587211 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:01.759399 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:01.758396 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:01.759399 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:01.758592 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:01.759399 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:01.758742 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 15:38:01.759399 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:01.758803 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm podName:b9bcd436-d777-4ca7-a840-bce674d247fd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:03.758786312 +0000 UTC m=+41.941948535 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g47gl" (UID: "b9bcd436-d777-4ca7-a840-bce674d247fd") : configmap "kube-rbac-proxy" not found Oct 30 15:38:01.759399 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:01.759226 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 15:38:01.761419 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:01.761389 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls podName:b9bcd436-d777-4ca7-a840-bce674d247fd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:03.759261771 +0000 UTC m=+41.942423999 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g47gl" (UID: "b9bcd436-d777-4ca7-a840-bce674d247fd") : secret "insights-runtime-extractor-tls" not found Oct 30 15:38:01.798209 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:01.786931 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:01.983919 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:01.983851 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:02.195085 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:02.195050 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:02.498970 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:02.498939 2572 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:38:02.499081 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:02.499017 2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:38:02.499462 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:02.499444 2572 scope.go:122] "RemoveContainer" containerID="f0bff60ba47b0b67c8bd6bf33c99bf64c08bc4973fb67c080a9b011c64cfbfd2" Oct 30 15:38:02.499632 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:02.499612 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tj9gm_openshift-console-operator(5306d4a8-24eb-461e-9384-f71ed3b41907)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" podUID="5306d4a8-24eb-461e-9384-f71ed3b41907" Oct 30 15:38:02.558780 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:02.558751 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:02.669171 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:02.669139 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-jzd98" event={"ID":"50c89e19-a6a1-4abe-906d-e2548c00e23a","Type":"ContainerStarted","Data":"78c295fc7d72d283535e3abf238b156913d56d8ed1fdf18333b02a68a9c2daff"} Oct 30 15:38:02.671472 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:02.671445 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fv7tg" event={"ID":"521c6b52-045a-447e-9d9e-49c0de21c0a2","Type":"ContainerStarted","Data":"0bf8baedd489ddf7863aad6c2c716128c0cc611964209e7f1b4407787709e9e0"} Oct 30 15:38:02.710593 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:02.710546 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-jzd98" podStartSLOduration=1.796445243 podStartE2EDuration="3.710509045s" totalImagesPullingTime="1.914063802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:37:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:02.70943533 +0000 UTC m=+40.892597567" watchObservedRunningTime="2026-10-30 15:38:02.710509045 +0000 UTC m=+40.893671282" Oct 30 15:38:02.749988 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:02.749901 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fv7tg" podStartSLOduration=1.866944942 podStartE2EDuration="3.749887305s" totalImagesPullingTime="1.882942363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:37:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:02.749223214 +0000 UTC m=+40.932385447" watchObservedRunningTime="2026-10-30 15:38:02.749887305 +0000 UTC m=+40.933049540" Oct 30 15:38:03.163685 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:03.163638 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:03.327600 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:03.327567 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:03.492638 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:03.492611 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:03.676870 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:03.676830 2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kpfss" event={"ID":"bbfe6e96-808d-4fb6-bce1-306f37eaf69d","Type":"ContainerStarted","Data":"f99d6beace583e717225abca88003cc87bd9d4a15e44612b0fcb631cbdae5a39"} Oct 30 15:38:03.725968 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:03.725906 2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kpfss" podStartSLOduration=33.730095295 podStartE2EDuration="40.725887336s" totalImagesPullingTime="6.995792041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:03.724410047 +0000 UTC m=+41.907572284" watchObservedRunningTime="2026-10-30 15:38:03.725887336 +0000 UTC m=+41.909049573" Oct 30 15:38:03.757425 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:03.757345 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:03.777449 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:03.777420 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:03.777562 ip-10-0-129-40 kubenswrapper[2572]: I1030 15:38:03.777548 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:03.777601 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:03.777583 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 15:38:03.777636 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:03.777623 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 15:38:03.777678 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:03.777650 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls podName:b9bcd436-d777-4ca7-a840-bce674d247fd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:07.777633373 +0000 UTC m=+45.960795592 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g47gl" (UID: "b9bcd436-d777-4ca7-a840-bce674d247fd") : secret "insights-runtime-extractor-tls" not found Oct 30 15:38:03.777678 ip-10-0-129-40 kubenswrapper[2572]: E1030 15:38:03.777672 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm podName:b9bcd436-d777-4ca7-a840-bce674d247fd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:07.777658083 +0000 UTC m=+45.960820320 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g47gl" (UID: "b9bcd436-d777-4ca7-a840-bce674d247fd") : configmap "kube-rbac-proxy" not found Oct 30 15:38:03.801373 ip-10-0-129-40 systemd[1]: Stopping Kubernetes Kubelet... Oct 30 15:38:03.807861 ip-10-0-129-40 systemd[1]: kubelet.service: Deactivated successfully. Oct 30 15:38:03.808024 ip-10-0-129-40 systemd[1]: Stopped Kubernetes Kubelet. Oct 30 15:38:03.808206 ip-10-0-129-40 systemd[1]: kubelet.service: Consumed 3.125s CPU time. Oct 30 15:38:03.817713 ip-10-0-129-40 systemd[1]: Starting Kubernetes Kubelet... Oct 30 15:38:03.915172 ip-10-0-129-40 kubenswrapper[5013]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 15:38:03.915172 ip-10-0-129-40 kubenswrapper[5013]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 15:38:03.915172 ip-10-0-129-40 kubenswrapper[5013]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 15:38:03.915172 ip-10-0-129-40 kubenswrapper[5013]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 15:38:03.917306 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917228 5013 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:38:03.917306 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917306 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917310 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917313 5013 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917316 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917319 5013 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917322 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917324 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917327 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917330 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917332 5013 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917335 5013 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917343 5013 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917348 5013 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917350 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917353 5013 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917356 5013 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917359 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917361 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917364 5013 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917366 5013 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:38:03.917379 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917369 5013 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917371 5013 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917374 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917376 5013 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917379 5013 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917382 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917384 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917387 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917390 5013 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917392 5013 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917395 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917398 5013 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917401 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917404 5013 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917406 5013 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917409 5013 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917411 5013 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917414 5013 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917416 5013 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917419 5013 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:38:03.917883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917421 5013 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917423 5013 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917426 5013 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917429 5013 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917431 5013 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917433 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917436 5013 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917438 5013 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917441 5013 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917443 5013 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917446 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917449 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917451 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917454 5013 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917457 5013 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917459 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917462 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917464 5013 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917467 5013 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917470 5013 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:38:03.918397 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917472 5013 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917474 5013 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917477 5013 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917479 5013 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917482 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917486 5013 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917490 5013 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917493 5013 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917497 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917499 5013 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917502 5013 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917505 5013 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917508 5013 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917510 5013 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917512 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917515 5013 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917517 5013 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917521 5013 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917525 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:38:03.918883 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917528 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917531 5013 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917535 5013 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917538 5013 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917544 5013 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917546 5013 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917549 5013 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917552 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917555 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917558 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917560 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917563 5013 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917565 5013 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917568 5013 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917570 5013 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917573 5013 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917575 5013 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917578 5013 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917580 5013 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:38:03.919375 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917583 5013 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917585 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917589 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917592 5013 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917594 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917597 5013 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.917600 5013 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918407 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918415 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918418 5013 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918421 5013 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918423 5013 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918426 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918429 5013 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918432 5013 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918434 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918437 5013 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918440 5013 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918443 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918445 5013 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:38:03.919868 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918448 5013 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918450 5013 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918453 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918456 5013 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918459 5013 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918462 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918465 5013 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918467 5013 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918470 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918472 5013 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918474 5013 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918477 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918479 5013 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918482 5013 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918484 5013 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918487 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918489 5013 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918491 5013 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918494 5013 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:38:03.920409 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918496 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918499 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918502 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918504 5013 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918508 5013 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918512 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918515 5013 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918518 5013 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918521 5013 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918524 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918526 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918529 5013 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918531 5013 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918534 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918536 5013 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918538 5013 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918541 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918544 5013 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918546 5013 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:38:03.920922 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918549 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918551 5013 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918553 5013 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918556 5013 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918558 5013 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918562 5013 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918566 5013 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918568 5013 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918571 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918574 5013 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918577 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918579 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918583 5013 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918585 5013 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918588 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918590 5013 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918593 5013 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918596 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918598 5013 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:38:03.921420 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918601 5013 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918603 5013 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918606 5013 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918608 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918611 5013 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918613 5013 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918616 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918618 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918621 5013 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918623 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918626 5013 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918629 5013 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918632 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918635 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918637 5013 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918640 5013 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918642 5013 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918645 5013 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918647 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918649 5013 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:38:03.921897 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918652 5013 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918654 5013 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918657 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918659 5013 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918662 5013 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918664 5013 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918666 5013 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918669 5013 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918672 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918674 5013 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918677 5013 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918679 5013 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918682 5013 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918685 5013 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918687 5013 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.918691 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919210 5013 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919217 5013 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919220 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919222 5013 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:38:03.922442 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919225 5013 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919228 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919230 5013 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919233 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919235 5013 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919238 5013 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919240 5013 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919243 5013 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919246 5013 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919249 5013 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919251 5013 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919254 5013 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919257 5013 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919259 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919262 5013 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919264 5013 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919269 5013 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919271 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919274 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919277 5013 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:38:03.922951 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919280 5013 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919283 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919286 5013 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919290 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919293 5013 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919295 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919297 5013 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919300 5013 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919302 5013 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919305 5013 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919307 5013 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919310 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919312 5013 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919315 5013 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919317 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919319 5013 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919322 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919325 5013 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919327 5013 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919329 5013 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:38:03.923497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919332 5013 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919334 5013 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919340 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919343 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919347 5013 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919352 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919355 5013 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919358 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919361 5013 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919364 5013 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919366 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919369 5013 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919372 5013 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919380 5013 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919383 5013 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919386 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919389 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919392 5013 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919394 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:38:03.924034 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919397 5013 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919399 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919402 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919404 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919406 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919409 5013 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919411 5013 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919414 5013 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919416 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919419 5013 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919421 5013 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919424 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919426 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919429 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919431 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919433 5013 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919436 5013 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919438 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919441 5013 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:38:03.924537 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919443 5013 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919446 5013 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919449 5013 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919452 5013 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919455 5013 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919457 5013 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919460 5013 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919463 5013 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919473 5013 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919477 5013 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919480 5013 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919483 5013 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919485 5013 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919488 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919490 5013 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919493 5013 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919495 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919498 5013 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919500 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919504 5013 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:38:03.925007 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919506 5013 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919509 5013 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919511 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.919513 5013 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919601 5013 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919608 5013 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919614 5013 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919619 5013 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919623 5013 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919626 5013 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919633 5013 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919637 5013 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919640 5013 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919643 5013 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919648 5013 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919651 5013 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919654 5013 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919657 5013 flags.go:64] FLAG: --cgroup-root="" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919660 5013 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919663 5013 flags.go:64] FLAG: --client-ca-file="" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919666 5013 flags.go:64] FLAG: --cloud-provider="external" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919670 5013 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919674 5013 flags.go:64] FLAG: --cluster-domain="" Oct 30 15:38:03.925519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919676 5013 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919679 5013 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919682 5013 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919686 5013 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919690 5013 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919692 5013 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919695 5013 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919698 5013 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919701 5013 flags.go:64] FLAG: --contention-profiling="false" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919704 5013 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919707 5013 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919710 5013 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919713 5013 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919717 5013 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919720 5013 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919723 5013 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919726 5013 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919729 5013 flags.go:64] FLAG: --enable-server="true" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919731 5013 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919736 5013 flags.go:64] FLAG: --event-burst="100" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919738 5013 flags.go:64] FLAG: --event-qps="50" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919742 5013 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919744 5013 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919749 5013 flags.go:64] FLAG: --eviction-hard="" Oct 30 15:38:03.926211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919753 5013 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919756 5013 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919759 5013 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919762 5013 flags.go:64] FLAG: --eviction-soft="" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919765 5013 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919768 5013 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919771 5013 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919774 5013 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919777 5013 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919780 5013 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919783 5013 flags.go:64] FLAG: --feature-gates="" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919787 5013 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919790 5013 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919793 5013 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919795 5013 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919798 5013 flags.go:64] FLAG: --healthz-port="10248" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919801 5013 flags.go:64] FLAG: --help="false" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919804 5013 flags.go:64] FLAG: --hostname-override="ip-10-0-129-40.ec2.internal" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919807 5013 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919810 5013 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919813 5013 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919816 5013 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919819 5013 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919822 5013 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 15:38:03.926806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919825 5013 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919828 5013 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919830 5013 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919833 5013 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919836 5013 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919839 5013 flags.go:64] FLAG: --kube-reserved="" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919842 5013 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919846 5013 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919849 5013 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919853 5013 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919856 5013 flags.go:64] FLAG: --lock-file="" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919858 5013 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919861 5013 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919864 5013 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919870 5013 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919872 5013 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919877 5013 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919880 5013 flags.go:64] FLAG: --logging-format="text" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919883 5013 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919886 5013 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919889 5013 flags.go:64] FLAG: --manifest-url="" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919892 5013 flags.go:64] FLAG: --manifest-url-header="" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919896 5013 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919899 5013 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919903 5013 flags.go:64] FLAG: --max-pods="110" Oct 30 15:38:03.927397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919906 5013 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919909 5013 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919912 5013 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919915 5013 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919918 5013 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919921 5013 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919923 5013 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919930 5013 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919933 5013 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919936 5013 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919939 5013 flags.go:64] FLAG: --pod-cidr="" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919942 5013 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919945 5013 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919948 5013 flags.go:64] FLAG: --pods-per-core="0" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919953 5013 flags.go:64] FLAG: --port="10250" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919956 5013 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919960 5013 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-022d59ab3c1157b41" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919963 5013 flags.go:64] FLAG: --qos-reserved="" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919966 5013 flags.go:64] FLAG: --read-only-port="10255" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919969 5013 flags.go:64] FLAG: --register-node="true" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919972 5013 flags.go:64] FLAG: --register-with-taints="" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919976 5013 flags.go:64] FLAG: --registry-burst="10" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919979 5013 flags.go:64] FLAG: --registry-qps="5" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919981 5013 flags.go:64] FLAG: --reserved-cpus="" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919988 5013 flags.go:64] FLAG: --reserved-memory="" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919992 5013 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 15:38:03.928301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919995 5013 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.919998 5013 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920001 5013 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920004 5013 flags.go:64] FLAG: --runonce="false" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920006 5013 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920009 5013 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920012 5013 flags.go:64] FLAG: --seccomp-default="false" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920015 5013 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920018 5013 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920021 5013 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920024 5013 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920027 5013 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920029 5013 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920032 5013 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920035 5013 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920037 5013 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920040 5013 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920043 5013 flags.go:64] FLAG: --system-cgroups="" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920046 5013 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920051 5013 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920056 5013 flags.go:64] FLAG: --tls-cert-file="" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920058 5013 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920063 5013 flags.go:64] FLAG: --tls-min-version="" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920066 5013 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920069 5013 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 15:38:03.929163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920072 5013 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920075 5013 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920077 5013 flags.go:64] FLAG: --v="2" Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920082 5013 flags.go:64] FLAG: --version="false" Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920086 5013 flags.go:64] FLAG: --vmodule="" Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920089 5013 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920104 5013 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920272 5013 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920276 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920279 5013 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920282 5013 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920284 5013 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920287 5013 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920289 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920292 5013 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920295 5013 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920297 5013 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920301 5013 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920304 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920308 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920310 5013 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:38:03.929842 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920313 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920315 5013 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920318 5013 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920321 5013 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920323 5013 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920326 5013 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920330 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920333 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920337 5013 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920339 5013 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920342 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920344 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920346 5013 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920349 5013 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920352 5013 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920354 5013 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920357 5013 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920359 5013 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920367 5013 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920370 5013 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:38:03.930394 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920373 5013 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920375 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920378 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920380 5013 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920382 5013 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920385 5013 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920387 5013 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920390 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920392 5013 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920394 5013 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920407 5013 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920410 5013 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920413 5013 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920415 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920418 5013 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920421 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920423 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920426 5013 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920430 5013 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920432 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:38:03.930913 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920437 5013 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920444 5013 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920449 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920454 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920458 5013 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920462 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920466 5013 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920469 5013 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920473 5013 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920477 5013 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920480 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920485 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920489 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920492 5013 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920495 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920497 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920501 5013 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920505 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920509 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:38:03.931469 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920513 5013 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920515 5013 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920518 5013 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920521 5013 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920523 5013 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920526 5013 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920528 5013 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920531 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920533 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920536 5013 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920539 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920542 5013 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920546 5013 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920549 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920552 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920554 5013 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920557 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920559 5013 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920562 5013 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920564 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:38:03.931943 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920567 5013 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920569 5013 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920572 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920575 5013 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920578 5013 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920581 5013 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920584 5013 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920586 5013 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920589 5013 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920591 5013 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920594 5013 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920596 5013 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920599 5013 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.920626 5013 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920669 5013 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920673 5013 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:38:03.932656 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920675 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920678 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920681 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920683 5013 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920686 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920688 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920690 5013 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920693 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920697 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920700 5013 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920703 5013 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920717 5013 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920722 5013 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920735 5013 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920740 5013 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920742 5013 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920745 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920747 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920750 5013 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:38:03.933402 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920752 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920755 5013 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920758 5013 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920761 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920764 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920767 5013 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920769 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920771 5013 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920774 5013 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920776 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920779 5013 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920781 5013 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920784 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920786 5013 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920789 5013 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920791 5013 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920794 5013 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920796 5013 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920798 5013 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920801 5013 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:38:03.933918 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920803 5013 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920807 5013 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920812 5013 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920814 5013 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920817 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920819 5013 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920822 5013 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920824 5013 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920827 5013 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920829 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920832 5013 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920834 5013 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920837 5013 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920841 5013 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920845 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920848 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920852 5013 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920854 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920857 5013 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:38:03.934460 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920859 5013 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920862 5013 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920865 5013 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920867 5013 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920870 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920872 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920875 5013 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920877 5013 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920880 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920882 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920884 5013 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920887 5013 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920890 5013 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920892 5013 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920897 5013 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920901 5013 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920903 5013 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920906 5013 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920908 5013 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:38:03.935081 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920911 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920913 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920916 5013 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920918 5013 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920920 5013 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920923 5013 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920926 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920928 5013 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920931 5013 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920933 5013 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920936 5013 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920939 5013 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920941 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920944 5013 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920946 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920949 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920951 5013 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920953 5013 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920956 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:38:03.935782 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920958 5013 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920963 5013 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920967 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920971 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920974 5013 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920976 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920979 5013 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.920981 5013 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.925804 5013 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.925823 5013 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.925847 5013 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.925854 5013 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925930 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925938 5013 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925943 5013 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925947 5013 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925951 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925955 5013 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925959 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:38:03.936364 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925963 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925968 5013 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925972 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925976 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925980 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925984 5013 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925988 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925992 5013 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.925996 5013 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926000 5013 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926004 5013 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926008 5013 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926012 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926016 5013 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926020 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926024 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926028 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926032 5013 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926036 5013 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926040 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:38:03.936832 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926044 5013 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926048 5013 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926054 5013 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926058 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926062 5013 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926066 5013 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926070 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926074 5013 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926078 5013 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926082 5013 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926086 5013 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926090 5013 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926094 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926098 5013 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926102 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926105 5013 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926109 5013 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926113 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926117 5013 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926121 5013 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:38:03.937361 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926126 5013 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926130 5013 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926134 5013 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926139 5013 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926143 5013 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926147 5013 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926151 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926156 5013 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926160 5013 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926163 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926167 5013 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926171 5013 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926175 5013 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926179 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926183 5013 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926187 5013 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926207 5013 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926212 5013 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926216 5013 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926220 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:38:03.937833 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926225 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926229 5013 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926233 5013 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926237 5013 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926241 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926244 5013 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926248 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926252 5013 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926257 5013 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926261 5013 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926265 5013 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926269 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926274 5013 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926278 5013 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926282 5013 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926286 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926290 5013 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926294 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926298 5013 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:38:03.938385 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926305 5013 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926310 5013 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926315 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926319 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926323 5013 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926327 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926331 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926337 5013 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926342 5013 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926348 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926354 5013 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926358 5013 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926362 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926365 5013 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926369 5013 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926373 5013 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926377 5013 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926381 5013 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926385 5013 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:38:03.938915 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926388 5013 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.926426 5013 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926622 5013 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926627 5013 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926632 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926636 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926641 5013 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926645 5013 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926649 5013 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926652 5013 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926656 5013 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926660 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926664 5013 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926669 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926672 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926676 5013 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:38:03.939548 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926680 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926684 5013 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926688 5013 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926692 5013 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926696 5013 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926700 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926704 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926709 5013 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926712 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926716 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926720 5013 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926724 5013 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926728 5013 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926732 5013 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926736 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926740 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926744 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926748 5013 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926751 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926755 5013 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:38:03.940014 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926759 5013 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926764 5013 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926768 5013 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926772 5013 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926776 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926779 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926783 5013 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926787 5013 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926791 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926795 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926799 5013 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926803 5013 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926829 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926835 5013 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926840 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926844 5013 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926849 5013 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926853 5013 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926857 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926861 5013 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:38:03.940635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926865 5013 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926869 5013 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926875 5013 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926881 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926886 5013 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926890 5013 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926894 5013 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926898 5013 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926902 5013 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926905 5013 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926909 5013 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926913 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926917 5013 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926922 5013 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926925 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926929 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926933 5013 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926937 5013 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926941 5013 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:38:03.941132 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926945 5013 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926949 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926953 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926957 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926961 5013 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926965 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926969 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926973 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926979 5013 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926985 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926989 5013 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926993 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.926997 5013 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927002 5013 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927006 5013 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927010 5013 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927014 5013 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927018 5013 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927022 5013 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:38:03.941672 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927026 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927030 5013 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927034 5013 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927038 5013 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927041 5013 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927045 5013 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927049 5013 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927054 5013 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927058 5013 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927061 5013 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927065 5013 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927069 5013 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927073 5013 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927077 5013 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.927112 5013 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927155 5013 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 15:38:03.942147 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927161 5013 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927166 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927170 5013 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927174 5013 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927178 5013 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927182 5013 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927186 5013 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927204 5013 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927210 5013 feature_gate.go:386] unrecognized feature gate: Example Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927213 5013 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927217 5013 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927221 5013 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927226 5013 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927230 5013 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927234 5013 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927238 5013 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927243 5013 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927246 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927250 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927254 5013 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 15:38:03.942563 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927258 5013 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927262 5013 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927266 5013 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927269 5013 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927273 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927277 5013 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927281 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927285 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927288 5013 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927292 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927296 5013 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927300 5013 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927304 5013 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927308 5013 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927312 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927316 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927320 5013 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927323 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927327 5013 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927331 5013 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 15:38:03.943101 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927335 5013 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927339 5013 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927343 5013 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927347 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927351 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927356 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927360 5013 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927364 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927368 5013 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927372 5013 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927376 5013 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927380 5013 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927384 5013 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927387 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927391 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927395 5013 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927400 5013 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927404 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927408 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927412 5013 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 15:38:03.943605 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927416 5013 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927420 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927424 5013 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927428 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927432 5013 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927436 5013 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927440 5013 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927443 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927447 5013 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927451 5013 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927455 5013 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927461 5013 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927466 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927470 5013 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927474 5013 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927478 5013 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927482 5013 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927486 5013 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927490 5013 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 15:38:03.944111 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927494 5013 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927499 5013 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927503 5013 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927507 5013 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927510 5013 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927514 5013 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927518 5013 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927522 5013 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927527 5013 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927530 5013 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927535 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927539 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927542 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927546 5013 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927550 5013 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927554 5013 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927558 5013 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927564 5013 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927569 5013 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 15:38:03.944663 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927578 5013 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927583 5013 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927587 5013 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927592 5013 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927596 5013 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927599 5013 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:03.927603 5013 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.927851 5013 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.929910 5013 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.929958 5013 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.930135 5013 server.go:1014] "Starting client certificate rotation" Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.930287 5013 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.930347 5013 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 15:32:23 +0000 UTC" deadline="2028-05-03 18:25:04.816240387 +0000 UTC" Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.930375 5013 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13226h47m0.885867825s" Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.930609 5013 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.931760 5013 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 15:38:03.945434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.932924 5013 log.go:25] "Validated CRI v1 runtime API" Oct 30 15:38:03.946058 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.934699 5013 log.go:25] "Validated CRI v1 image API" Oct 30 15:38:03.946058 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.935489 5013 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 15:38:03.946058 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.937569 5013 fs.go:135] Filesystem UUIDs: map[2656c423-a1d4-4b52-ae84-13ec73be3fb6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ac001b9c-0cdf-4e20-99dc-7e71067bfab3:/dev/nvme0n1p4] Oct 30 15:38:03.946231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.937590 5013 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/10f4399d852063599dc1f924ddbc866e3666ff6e11a68a84b5b0443d9e95587f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/10f4399d852063599dc1f924ddbc866e3666ff6e11a68a84b5b0443d9e95587f/userdata/shm major:0 minor:83 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/178c5d7f1cec252740eeb224e34e1b09e1f276bd51b71559cc18b9910a859e83/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/178c5d7f1cec252740eeb224e34e1b09e1f276bd51b71559cc18b9910a859e83/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/193d22c8fdd83f86ce36aa18cf132e1cae590150b0cbbe662f18bcf9f4d4f438/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/193d22c8fdd83f86ce36aa18cf132e1cae590150b0cbbe662f18bcf9f4d4f438/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/26d5211a3f51660e7ad2eaa8a951e817188cc34d7f0f72f34b9d342b9fdca355/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/26d5211a3f51660e7ad2eaa8a951e817188cc34d7f0f72f34b9d342b9fdca355/userdata/shm major:0 minor:89 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/28b815f8ae9435007e7952a029539da96090d444abfe27739689310371565426/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/28b815f8ae9435007e7952a029539da96090d444abfe27739689310371565426/userdata/shm major:0 minor:326 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/29aedf8a9de47ad4db2cef7e3c96d8a34b6e7dd5e05376a6b5c2e958063947bd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/29aedf8a9de47ad4db2cef7e3c96d8a34b6e7dd5e05376a6b5c2e958063947bd/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3c3fff7263bb1bbea6d51328fa0da0d1ebce890bc0664b6d54c459b03df4ce54/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3c3fff7263bb1bbea6d51328fa0da0d1ebce890bc0664b6d54c459b03df4ce54/userdata/shm major:0 minor:347 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4beedffb5268777c1fc11108dd5832c033e8373f74c9c8f010e445cf7bc86669/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4beedffb5268777c1fc11108dd5832c033e8373f74c9c8f010e445cf7bc86669/userdata/shm major:0 minor:349 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5e17aba9bf8cf3d0b3ce4706701e0c0bcddf8f4a9092b910c642eca0dc7e705e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5e17aba9bf8cf3d0b3ce4706701e0c0bcddf8f4a9092b910c642eca0dc7e705e/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5f96514100a9ea2337b5b1724185f3e4afa30bca37a086217079d652803e0403/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5f96514100a9ea2337b5b1724185f3e4afa30bca37a086217079d652803e0403/userdata/shm major:0 minor:293 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/63907f7263857348468b35543dbbe08956cfdd4ef04e6bbf12705dac4b29d34e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/63907f7263857348468b35543dbbe08956cfdd4ef04e6bbf12705dac4b29d34e/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/639b913c660246019f02aba1039b0e529e9615ee8d2d6190a6c690eec0078d97/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/639b913c660246019f02aba1039b0e529e9615ee8d2d6190a6c690eec0078d97/userdata/shm major:0 minor:298 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7100b883ded8512c8facba4245b083a52fb9801e6563e0a7958007b9904f4b5b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7100b883ded8512c8facba4245b083a52fb9801e6563e0a7958007b9904f4b5b/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7d429a794704486ee79bdbc437afe85d138f2bffdcd0d2cc6eed39c04d03250c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7d429a794704486ee79bdbc437afe85d138f2bffdcd0d2cc6eed39c04d03250c/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/801d2f35ec081124f933cca6d52aab3efebc43063b61af2c3ceb3f61e47000cb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/801d2f35ec081124f933cca6d52aab3efebc43063b61af2c3ceb3f61e47000cb/userdata/shm major:0 minor:220 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/85a01e79e10328907cc525f8fb341eefe4898bdd62ff4a885933fcd376ee6fc7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/85a01e79e10328907cc525f8fb341eefe4898bdd62ff4a885933fcd376ee6fc7/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a83821ca18e0e2b64d3caa415f922b5b97cfc1c78eaf1787ff1b0e4699b130d6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a83821ca18e0e2b64d3caa415f922b5b97cfc1c78eaf1787ff1b0e4699b130d6/userdata/shm major:0 minor:231 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/adb395cb5008e7e50add661ae224bf0d113a15a629ade7d908f70c8f5118805f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/adb395cb5008e7e50add661ae224bf0d113a15a629ade7d908f70c8f5118805f/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b2a08d9a67abde77eea0080daa2a5bb8250c0e80fe9eeec12c657eee6d67ed78/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b2a08d9a67abde77eea0080daa2a5bb8250c0e80fe9eeec12c657eee6d67ed78/userdata/shm major:0 minor:91 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b9b4add9e6248070ae9e2de7d0e34209fef2c40ab9812dd1e22ea756052e6699/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b9b4add9e6248070ae9e2de7d0e34209fef2c40ab9812dd1e22ea756052e6699/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c13800af7a74e91e88669cd2a1bf00e88ae1c7a7292b5e83e79ab30f3df07a41/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c13800af7a74e91e88669cd2a1bf00e88ae1c7a7292b5e83e79ab30f3df07a41/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/de891953be9ade7bbb17c6b8e7cee8871cb4b970879f0a38f6acd4642a740442/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/de891953be9ade7bbb17c6b8e7cee8871cb4b970879f0a38f6acd4642a740442/userdata/shm major:0 minor:222 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/edda3baa71c3486c5355cca87b5a31e02e178daa1d6cdb2a3bb497d97e64094f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/edda3baa71c3486c5355cca87b5a31e02e178daa1d6cdb2a3bb497d97e64094f/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eedd95ddefbd506f780979180a6db801a452bca507e96ddffe75b29b555f15c1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eedd95ddefbd506f780979180a6db801a452bca507e96ddffe75b29b555f15c1/userdata/shm major:0 minor:227 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f8f31e26c79f285a251c441f8180d9c32b4f29e9ee9f1e9509da00d9a6f36be1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f8f31e26c79f285a251c441f8180d9c32b4f29e9ee9f1e9509da00d9a6f36be1/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0025b75d-4f3c-4c5a-91ca-505bda44bccb/volumes/kubernetes.io~projected/kube-api-access-hsp5p:{mountpoint:/var/lib/kubelet/pods/0025b75d-4f3c-4c5a-91ca-505bda44bccb/volumes/kubernetes.io~projected/kube-api-access-hsp5p major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a1bfd21-ddfa-4820-8356-49214c4cfc3b/volumes/kubernetes.io~projected/kube-api-access-t9dtn:{mountpoint:/var/lib/kubelet/pods/0a1bfd21-ddfa-4820-8356-49214c4cfc3b/volumes/kubernetes.io~projected/kube-api-access-t9dtn major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a1bfd21-ddfa-4820-8356-49214c4cfc3b/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/0a1bfd21-ddfa-4820-8356-49214c4cfc3b/volumes/kubernetes.io~secret/default-certificate major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a1bfd21-ddfa-4820-8356-49214c4cfc3b/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/0a1bfd21-ddfa-4820-8356-49214c4cfc3b/volumes/kubernetes.io~secret/stats-auth major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/14a2267f-e095-45cb-b0bc-99d7ab20611a/volumes/kubernetes.io~projected/kube-api-access-lvdcn:{mountpoint:/var/lib/kubelet/pods/14a2267f-e095-45cb-b0bc-99d7ab20611a/volumes/kubernetes.io~projected/kube-api-access-lvdcn major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13/volumes/kubernetes.io~projected/kube-api-access-zlmrz:{mountpoint:/var/lib/kubelet/pods/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13/volumes/kubernetes.io~projected/kube-api-access-zlmrz major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec/volumes/kubernetes.io~projected/kube-api-access-n69wr:{mountpoint:/var/lib/kubelet/pods/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec/volumes/kubernetes.io~projected/kube-api-access-n69wr major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec/volumes/kubernetes.io~secret/serving-cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e/volumes/kubernetes.io~projected/kube-api-access-jxv8s:{mountpoint:/var/lib/kubelet/pods/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e/volumes/kubernetes.io~projected/kube-api-access-jxv8s major:0 minor:303 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e/volumes/kubernetes.io~secret/signing-key major:0 minor:302 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2d1df50d-be8d-4d79-ae09-2eb0853f313d/volumes/kubernetes.io~projected/kube-api-access-z6dbr:{mountpoint:/var/lib/kubelet/pods/2d1df50d-be8d-4d79-ae09-2eb0853f313d/volumes/kubernetes.io~projected/kube-api-access-z6dbr major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2d95d50f-c563-4080-909a-8ec8dc3b69b7/volumes/kubernetes.io~projected/kube-api-access-8zdpn:{mountpoint:/var/lib/kubelet/pods/2d95d50f-c563-4080-909a-8ec8dc3b69b7/volumes/kubernetes.io~projected/kube-api-access-8zdpn major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2d95d50f-c563-4080-909a-8ec8dc3b69b7/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/2d95d50f-c563-4080-909a-8ec8dc3b69b7/volumes/kubernetes.io~secret/serving-cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30238695-c5e1-433b-b666-c822522e05e2/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/30238695-c5e1-433b-b666-c822522e05e2/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30238695-c5e1-433b-b666-c822522e05e2/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/30238695-c5e1-433b-b666-c822522e05e2/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30238695-c5e1-433b-b666-c822522e05e2/volumes/kubernetes.io~projected/kube-api-access-zqmfg:{mountpoint:/var/lib/kubelet/pods/30238695-c5e1-433b-b666-c822522e05e2/volumes/kubernetes.io~projected/kube-api-access-zqmfg major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3173666e-bb4d-4f89-99bd-061c19370b91/volumes/kubernetes.io~projected/kube-api-access-67j8n:{mountpoint:/var/lib/kubelet/pods/3173666e-bb4d-4f89-99bd-061c19370b91/volumes/kubernetes.io~projected/kube-api-access-67j8n major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4345f2dd-427e-40d1-a5cc-43440c8cd4f0/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/4345f2dd-427e-40d1-a5cc-43440c8cd4f0/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4345f2dd-427e-40d1-a5cc-43440c8cd4f0/volumes/kubernetes.io~projected/kube-api-access-tbkl9:{mountpoint:/var/lib/kubelet/pods/4345f2dd-427e-40d1-a5cc-43440c8cd4f0/volumes/kubernetes.io~projected/kube-api-access-tbkl9 major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4345f2dd-427e-40d1-a5cc-43440c8cd4f0/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/4345f2dd-427e-40d1-a5cc-43440c8cd4f0/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4345f2dd-427e-40d1-a5cc-43440c8cd4f0/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/4345f2dd-427e-40d1-a5cc-43440c8cd4f0/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/50c89e19-a6a1-4abe-906d-e2548c00e23a/volumes/kubernetes.io~projected/kube-api-access-72qnj:{mountpoint:/var/lib/kubelet/pods/50c89e19-a6a1-4abe-906d-e2548c00e23a/volumes/kubernetes.io~projected/kube-api-access-72qnj major:0 minor:336 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/521c6b52-045a-447e-9d9e-49c0de21c0a2/volumes/kubernetes.io~projected/kube-api-access-snbvb:{mountpoint:/var/lib/kubelet/pods/521c6b52-045a-447e-9d9e-49c0de21c0a2/volumes/kubernetes.io~projected/kube-api-access-snbvb major:0 minor:337 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5306d4a8-24eb-461e-9384-f71ed3b41907/volumes/kubernetes.io~projected/kube-api-access-ccndt:{mountpoint:/var/lib/kubelet/pods/5306d4a8-24eb-461e-9384-f71ed3b41907/volumes/kubernetes.io~projected/kube-api-access-ccndt major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5306d4a8-24eb-461e-9384-f71ed3b41907/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/5306d4a8-24eb-461e-9384-f71ed3b41907/volumes/kubernetes.io~secret/serving-cert major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7df10531-4deb-4b95-ba47-9fb316775878/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/7df10531-4deb-4b95-ba47-9fb316775878/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/910e7c90-9822-46c7-aa83-929c32dee95c/volumes/kubernetes.io~projected/kube-api-access-jl229:{mountpoint:/var/lib/kubelet/pods/910e7c90-9822-46c7-aa83-929c32dee95c/volumes/kubernetes.io~projected/kube-api-access-jl229 major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/910e7c90-9822-46c7-aa83-929c32dee95c/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/910e7c90-9822-46c7-aa83-929c32dee95c/volumes/kubernetes.io~secret/cert major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91393a38-a119-4479-adb6-f8fe6316bce3/volumes/kubernetes.io~projected/kube-api-access-vn2bz:{mountpoint:/var/lib/kubelet/pods/91393a38-a119-4479-adb6-f8fe6316bce3/volumes/kubernetes.io~projected/kube-api-access-vn2bz major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/92898f00-aba8-475c-9389-1f4d3fae8103/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/92898f00-aba8-475c-9389-1f4d3fae8103/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/92898f00-aba8-475c-9389-1f4d3fae8103/volumes/kubernetes.io~projected/kube-api-access-v8xdt:{mountpoint:/var/lib/kubelet/pods/92898f00-aba8-475c-9389-1f4d3fae8103/volumes/kubernetes.io~projected/kube-api-access-v8xdt major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/92898f00-aba8-475c-9389-1f4d3fae8103/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/92898f00-aba8-475c-9389-1f4d3fae8103/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/92898f00-aba8-475c-9389-1f4d3fae8103/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/92898f00-aba8-475c-9389-1f4d3fae8103/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9c690dd2-7aa3-46c8-981b-0cc314fc2889/volumes/kubernetes.io~projected/kube-api-access-zr8pl:{mountpoint:/var/lib/kubelet/pods/9c690dd2-7aa3-46c8-981b-0cc314fc2889/volumes/kubernetes.io~projected/kube-api-access-zr8pl major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9eeaa1d3-abc9-4811-acad-f3a4419d967e/volumes/kubernetes.io~projected/kube-api-access-g95zs:{mountpoint:/var/lib/kubelet/pods/9eeaa1d3-abc9-4811-acad-f3a4419d967e/volumes/kubernetes.io~projected/kube-api-access-g95zs major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b9bcd436-d777-4ca7-a840-bce674d247fd/volumes/kubernetes.io~projected/kube-api-access-dj24q:{mountpoint:/var/lib/kubelet/pods/b9bcd436-d777-4ca7-a840-bce674d247fd/volumes/kubernetes.io~projected/kube-api-access-dj24q major:0 minor:350 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bbfe6e96-808d-4fb6-bce1-306f37eaf69d/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/bbfe6e96-808d-4fb6-bce1-306f37eaf69d/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bc73f5d0-671f-4cf5-8aab-325db0b149e7/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/bc73f5d0-671f-4cf5-8aab-325db0b149e7/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bc73f5d0-671f-4cf5-8aab-325db0b149e7/volumes/kubernetes.io~projected/kube-api-access-jncfk:{mountpoint:/var/lib/kubelet/pods/bc73f5d0-671f-4cf5-8aab-325db0b149e7/volumes/kubernetes.io~projected/kube-api-access-jncfk major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bc73f5d0-671f-4cf5-8aab-325db0b149e7/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/bc73f5d0-671f-4cf5-8aab-325db0b149e7/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c04fe7dc-a0c3-4626-8512-a519e8821681/volumes/kubernetes.io~projected/kube-api-access-bjvgh:{mountpoint:/var/lib/kubelet/pods/c04fe7dc-a0c3-4626-8512-a519e8821681/volumes/kubernetes.io~projected/kube-api-access-bjvgh major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c134f615-1a89-4c6a-9540-62846db76aa0/volumes/kubernetes.io~projected/kube-api-access-gxsbg:{mountpoint:/var/lib/kubelet/pods/c134f615-1a89-4c6a-9540-62846db76aa0/volumes/kubernetes.io~projected/kube-api-access-gxsbg major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d/volumes/kubernetes.io~projected/kube-api-access-vsnqq:{mountpoint:/var/lib/kubelet/pods/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d/volumes/kubernetes.io~projected/kube-api-access-vsnqq major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf86f01f-c44d-46a3-be13-03c99b006285/volumes/kubernetes.io~projected/kube-api-access-zrwsv:{mountpoint:/var/lib/kubelet/pods/cf86f01f-c44d-46a3-be13-03c99b006285/volumes/kubernetes.io~projected/kube-api-access-zrwsv major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e54831c9-77f6-4bce-b8fc-bbfd1838dddd/volumes/kubernetes.io~projected/kube-api-access-dchj2:{mountpoint:/var/lib/kubelet/pods/e54831c9-77f6-4bce-b8fc-bbfd1838dddd/volumes/kubernetes.io~projected/kube-api-access-dchj2 major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ea29f558-cb33-4c42-9663-5db5361081b1/volumes/kubernetes.io~projected/kube-api-access-b56wc:{mountpoint:/var/lib/kubelet/pods/ea29f558-cb33-4c42-9663-5db5361081b1/volumes/kubernetes.io~projected/kube-api-access-b56wc major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb4bb521-d604-4dbe-b464-46fb320253a5/volumes/kubernetes.io~projected/kube-api-access-njfdc:{mountpoint:/var/lib/kubelet/pods/eb4bb521-d604-4dbe-b464-46fb320253a5/volumes/kubernetes.io~projected/kube-api-access-njfdc major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb4bb521-d604-4dbe-b464-46fb320253a5/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/eb4bb521-d604-4dbe-b464-46fb320253a5/volumes/kubernetes.io~secret/serving-cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eee942d8-9866-4315-ab29-d721cf68455c/volumes/kubernetes.io~projected/kube-api-access-7twh7:{mountpoint:/var/lib/kubelet/pods/eee942d8-9866-4315-ab29-d721cf68455c/volumes/kubernetes.io~projected/kube-api-access-7twh7 major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f77495d2-eb88-48cb-b46e-510fe055471b/volumes/kubernetes.io~projected/kube-api-access-kmfvd:{mountpoint:/var/lib/kubelet/pods/f77495d2-eb88-48cb-b46e-510fe055471b/volumes/kubernetes.io~projected/kube-api-access-kmfvd major:0 minor:237 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/809f0d3b43555b276b1443ee8eb1e3a84ba101904facb42d64e09b526cec4079/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/f28a070822e00754cc4a884f41a4c4eeb3006e411f092d84a9e627eb6ee756d9/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/6d1b6785bc06858ec54e6e3a3daad1d696e16c221e68c0db51b55d399c715c96/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/061e214275fce7561c6927dc61193be72e54660ece4c1e60e40c9945c84887fb/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/2422485541ddd2e74900b9c5f3746f27d09f88a0f3c4f6bf91cebde4a4cd9ce9/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/b0f4585dd678521123423915cc6e433057864c7341c1c7da349ce3010746e963/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/b689a8aa5169cc5abea92fca369c8359089152820c535a206c9c046664c5962e/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/749aab5caa4c8776d1803b001900321a41a35a9edbe438d2f87fc561f585697b/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/57360d8165228de9242139e6206d336debc4469344c793b12fa01099b0b56c5e/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/a958c07083d9008582da679b6c935e475fae9526a1efe3bf90e2a96f3b1c8ec7/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/1fc6f18d72141dd62ad19532bd83622440f1b1ceb7617b9562cf0182259fd86a/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-148:{mountpoint:/var/lib/containers/storage/overlay/9bd04a2740124923ebee59b4ea81932827c9996e3bb532819fc48bebbb026e24/merged major:0 minor:148 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/8989a1fac583af2f63da86f2a2a9a64b226fe95a94273cc2433cd62b3903e597/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/78a03e4feff974ad9dd39daf8378781acd779d15182ea1008e13b43947ad8f50/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/7d95fc7c2ef6f7a68827f099b2ead192ba1d3ffb3ae3da33a61d7661140e7c1c/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/3a791097befc311035cd89afa4da50c3bb562886e378c6e01eb542b7ea27b2fd/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/5fbb9af754f235ed813954c801d824ffe1a263624bea87f9e44f3aadc492143c/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/5fd4a7933b2df06b7dd826b773f2a9e22194e871927acf236efbfba1459d0f50/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/a554bb48b62cbdf2b70f6620417dac35992b3f830dabe046e232c11d7489dd20/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/9b6402fb8cd49e36e1dbff43b0c1daebdeb9ef62a9d6a1cd29efd3c6e1d2341f/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/dc30c201bc6b7b52c10bd31e65536cd49f4f291b87b34acd25be6dd2050a81ec/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/95f4fe8c72af6eb044bc89973011499009c37fc2f70b77ad3b6356bcbf755d67/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/c5337d7c957b476ac33288a2aca2c256ffbde897789d9fbb3850b489c6be4e0a/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/38a9aaebd5997e44a525d3c94baa21e1581164b250ec5e1b2232a3bcb5032f08/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/61ec28d75db2bd031fd7efac91437b66d06a03fd54edc6ba932ee01c31ad2838/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/0738ee7e4dfde45f6e801997e8759d22a4962f38aef5b48d762cac20be70cafb/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/97490f94a5949e4fb33e6a1a7d20d83e80091a6aca1dcf55306c701496850b3c/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/d4233223348958acf2336227015a492fc8ba121874e279569a8d09c7e977fc64/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/e84b4ef73149677f3e9134522fc0e276ab0db4febbed3335dbfbe218dd738f5f/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-267:{mountpoint:/var/lib/containers/storage/overlay/e8f21d5ab53aec293aaa2a072ddf43a59204f0482f42eaafd611dd51f185ba46/merged major:0 minor:267 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/ecbf556e4bb2b75b07c8e1082f45b3c7eb0afd89e282d6df7aff1a0dcaa09b69/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/ba58a6092c4ae95f42d3bce302ed80fbfa6bb318a6e08dad76fe326554ba0307/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/c3b37f69e0c27da8e77d3f09625d2515143010f01edf09263d405cf7dc127693/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-280:{mountpoint:/var/lib/containers/storage/overlay/a8c6854d7cf8ef552a1bafc5d0d08c158825b13a392c449cfddf107c7e084603/merged major:0 minor:280 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/276cca8fc34d1acf296628748a8f402cf87458f41ee2b9fa1835603067f67c74/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/3fea3013599f0d01a34a65fb6267c40a34fe7f1172a01d3d3e1d94a06273c464/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/adbe6828f637bb3ec81eb29b24250d82bea72aa00cb8371588c3bad80feef38d/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/dd031fa96cc56fe8b37ceec5754552f0183968225b20059d67cd3d310555d33e/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/8a487d7706d357c1326810c01c83d0d2e97694ccee8d3a64d914cdcae40e09dc/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/011f6f6e341d32e1fac541e5984d0fe977cc288348eb20ac308e5e3d19929739/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/386c61324571b66c1595baf07616f34192f43bb5013a2670df1d14ebf276e019/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/14b7a10f1b23f53dae71028c6fa907d5269d1aef425d36bd1f4b390ec2148566/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/814467cb7db7f6679f1876ddd6e0c9f71e0af90209897384ffaba5ec82f4d27b/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-340:{mountpoint:/var/lib/containers/storage/overlay/7bd919920ce02e249c99e4e7cddf8a0355fb9d9ad8608c07346f9957dac10545/merged major:0 minor:340 fsType:overlay blockSize:0} overlay_0-351:{mountpoint:/var/lib/containers/storage/overlay/1159b3ffbe427f11656d58785edb62fde43430d6d9795a459fb244fa6433f621/merged major:0 minor:351 fsType:overlay blockSize:0} overlay_0-353:{mountpoint:/var/lib/containers/storage/overlay/8131cbb81a144bc7c5eea2351db74cfa91d4aeaa0e8b7a45878e4126f7e8a20a/merged major:0 minor:353 fsType:overlay blockSize:0} overlay_0-358:{mountpoint:/var/lib/containers/storage/overlay/c96c7ad085a82b74be57ac972832dd55761c3db38c4d9c0b6843527c12c5e6c6/merged major:0 minor:358 fsType:overlay blockSize:0} overlay_0-361:{mountpoint:/var/lib/containers/storage/overlay/c8883be9feb31d4c5a8ac7d8521fea794a3b66ea5ff6845900b14aaecef81878/merged major:0 minor:361 fsType:overlay blockSize:0} overlay_0-363:{mountpoint:/var/lib/containers/storage/overlay/4ff431dd9fcdefd30bb6043c709fc842cda9b3795e057d78a83d51aabe1a453e/merged major:0 minor:363 fsType:overlay blockSize:0} overlay_0-365:{mountpoint:/var/lib/containers/storage/overlay/edea836b2962327b219516b9eb92be522eab6db72a4b1e8ea93028c6e64301f2/merged major:0 minor:365 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/cffa5aa7a6f579e516d868e1e87de34d28ade24a1c719d3fbd6bea40ae001b65/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/d6cc1a4a75c80267533c07a8cb55fb041723ed94a7bb0de62c5d4862c6ca2a00/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/635859c91c7859a9639edbcdd072ba1ebe97b107f5c820e7e090f20775c7b6e0/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/cf773209812d67d2f31f920d7a222628a295598bd245ba07018c6d22d2dca932/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/aaf0f613967cf9dca166263d797ff9d373c7f92dc2f3317e184b981b2105da9e/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/ce9b24871829ef884f601255768806ea3fbb16eba7a9386e64fedfdeca183cfb/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-85:{mountpoint:/var/lib/containers/storage/overlay/d05c86214444d44d7438947c5c52a18f7759af324a355321b2aa210349806ab1/merged major:0 minor:85 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/def7ef6872c90034f85528f6a43d4051118f21db6f5e7149f3c45a31c54759fd/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/dbe00d3538c38927adf8e34b779d8920108b624f0b2a42050972fbb4cd319ded/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/7c01b3d4bf1f07fa98d1bc83787d96919441b9068e80c96799b90b456187c431/merged major:0 minor:98 fsType:overlay blockSize:0}] Oct 30 15:38:03.946835 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.945723 5013 manager.go:217] Machine: {Timestamp:2026-10-30 15:38:03.945010797 +0000 UTC m=+0.095907819 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100396 MemoryCapacity:32812007424 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2694473106a743752da8d61b2e0e31 SystemUUID:ec269447-3106-a743-752d-a8d61b2e0e31 BootID:bbf8ae9a-0b0e-4439-9c35-75ea962e3635 Filesystems:[{Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2d1df50d-be8d-4d79-ae09-2eb0853f313d/volumes/kubernetes.io~projected/kube-api-access-z6dbr DeviceMajor:0 DeviceMinor:210 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/92898f00-aba8-475c-9389-1f4d3fae8103/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:215 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/30238695-c5e1-433b-b666-c822522e05e2/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/cf86f01f-c44d-46a3-be13-03c99b006285/volumes/kubernetes.io~projected/kube-api-access-zrwsv DeviceMajor:0 DeviceMinor:63 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13/volumes/kubernetes.io~projected/kube-api-access-zlmrz DeviceMajor:0 DeviceMinor:62 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/91393a38-a119-4479-adb6-f8fe6316bce3/volumes/kubernetes.io~projected/kube-api-access-vn2bz DeviceMajor:0 DeviceMinor:67 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/193d22c8fdd83f86ce36aa18cf132e1cae590150b0cbbe662f18bcf9f4d4f438/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/0a1bfd21-ddfa-4820-8356-49214c4cfc3b/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:226 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-365 DeviceMajor:0 DeviceMinor:365 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/92898f00-aba8-475c-9389-1f4d3fae8103/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:201 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-340 DeviceMajor:0 DeviceMinor:340 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/10f4399d852063599dc1f924ddbc866e3666ff6e11a68a84b5b0443d9e95587f/userdata/shm DeviceMajor:0 DeviceMinor:83 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4345f2dd-427e-40d1-a5cc-43440c8cd4f0/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:202 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:197 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/4345f2dd-427e-40d1-a5cc-43440c8cd4f0/volumes/kubernetes.io~projected/kube-api-access-tbkl9 DeviceMajor:0 DeviceMinor:214 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e/volumes/kubernetes.io~projected/kube-api-access-jxv8s DeviceMajor:0 DeviceMinor:303 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0a1bfd21-ddfa-4820-8356-49214c4cfc3b/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:224 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d/volumes/kubernetes.io~projected/kube-api-access-vsnqq DeviceMajor:0 DeviceMinor:61 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/5306d4a8-24eb-461e-9384-f71ed3b41907/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:204 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/910e7c90-9822-46c7-aa83-929c32dee95c/volumes/kubernetes.io~projected/kube-api-access-jl229 DeviceMajor:0 DeviceMinor:235 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/bbfe6e96-808d-4fb6-bce1-306f37eaf69d/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:288 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/639b913c660246019f02aba1039b0e529e9615ee8d2d6190a6c690eec0078d97/userdata/shm DeviceMajor:0 DeviceMinor:298 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406003712 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/14a2267f-e095-45cb-b0bc-99d7ab20611a/volumes/kubernetes.io~projected/kube-api-access-lvdcn DeviceMajor:0 DeviceMinor:65 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/ea29f558-cb33-4c42-9663-5db5361081b1/volumes/kubernetes.io~projected/kube-api-access-b56wc DeviceMajor:0 DeviceMinor:213 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/9eeaa1d3-abc9-4811-acad-f3a4419d967e/volumes/kubernetes.io~projected/kube-api-access-g95zs DeviceMajor:0 DeviceMinor:216 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b2a08d9a67abde77eea0080daa2a5bb8250c0e80fe9eeec12c657eee6d67ed78/userdata/shm DeviceMajor:0 DeviceMinor:91 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eb4bb521-d604-4dbe-b464-46fb320253a5/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:206 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4beedffb5268777c1fc11108dd5832c033e8373f74c9c8f010e445cf7bc86669/userdata/shm DeviceMajor:0 DeviceMinor:349 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/30238695-c5e1-433b-b666-c822522e05e2/volumes/kubernetes.io~projected/kube-api-access-zqmfg DeviceMajor:0 DeviceMinor:64 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-148 DeviceMajor:0 DeviceMinor:148 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/178c5d7f1cec252740eeb224e34e1b09e1f276bd51b71559cc18b9910a859e83/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5f96514100a9ea2337b5b1724185f3e4afa30bca37a086217079d652803e0403/userdata/shm DeviceMajor:0 DeviceMinor:293 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/521c6b52-045a-447e-9d9e-49c0de21c0a2/volumes/kubernetes.io~projected/kube-api-access-snbvb DeviceMajor:0 DeviceMinor:337 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/edda3baa71c3486c5355cca87b5a31e02e178daa1d6cdb2a3bb497d97e64094f/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5306d4a8-24eb-461e-9384-f71ed3b41907/volumes/kubernetes.io~projected/kube-api-access-ccndt DeviceMajor:0 DeviceMinor:212 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/eee942d8-9866-4315-ab29-d721cf68455c/volumes/kubernetes.io~projected/kube-api-access-7twh7 DeviceMajor:0 DeviceMinor:218 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/0a1bfd21-ddfa-4820-8356-49214c4cfc3b/volumes/kubernetes.io~projected/kube-api-access-t9dtn DeviceMajor:0 DeviceMinor:232 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/63907f7263857348468b35543dbbe08956cfdd4ef04e6bbf12705dac4b29d34e/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/2d95d50f-c563-4080-909a-8ec8dc3b69b7/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:203 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/92898f00-aba8-475c-9389-1f4d3fae8103/volumes/kubernetes.io~projected/kube-api-access-v8xdt DeviceMajor:0 DeviceMinor:209 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/0025b75d-4f3c-4c5a-91ca-505bda44bccb/volumes/kubernetes.io~projected/kube-api-access-hsp5p DeviceMajor:0 DeviceMinor:66 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eb4bb521-d604-4dbe-b464-46fb320253a5/volumes/kubernetes.io~projected/kube-api-access-njfdc DeviceMajor:0 DeviceMinor:217 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/bc73f5d0-671f-4cf5-8aab-325db0b149e7/volumes/kubernetes.io~projected/kube-api-access-jncfk DeviceMajor:0 DeviceMinor:60 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/92898f00-aba8-475c-9389-1f4d3fae8103/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:205 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eedd95ddefbd506f780979180a6db801a452bca507e96ddffe75b29b555f15c1/userdata/shm DeviceMajor:0 DeviceMinor:227 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-358 DeviceMajor:0 DeviceMinor:358 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-363 DeviceMajor:0 DeviceMinor:363 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7100b883ded8512c8facba4245b083a52fb9801e6563e0a7958007b9904f4b5b/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/26d5211a3f51660e7ad2eaa8a951e817188cc34d7f0f72f34b9d342b9fdca355/userdata/shm DeviceMajor:0 DeviceMinor:89 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/bc73f5d0-671f-4cf5-8aab-325db0b149e7/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/30238695-c5e1-433b-b666-c822522e05e2/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/2d95d50f-c563-4080-909a-8ec8dc3b69b7/volumes/kubernetes.io~projected/kube-api-access-8zdpn DeviceMajor:0 DeviceMinor:207 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f8f31e26c79f285a251c441f8180d9c32b4f29e9ee9f1e9509da00d9a6f36be1/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/b9bcd436-d777-4ca7-a840-bce674d247fd/volumes/kubernetes.io~projected/kube-api-access-dj24q DeviceMajor:0 DeviceMinor:350 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/7df10531-4deb-4b95-ba47-9fb316775878/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/50c89e19-a6a1-4abe-906d-e2548c00e23a/volumes/kubernetes.io~projected/kube-api-access-72qnj DeviceMajor:0 DeviceMinor:336 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c13800af7a74e91e88669cd2a1bf00e88ae1c7a7292b5e83e79ab30f3df07a41/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/801d2f35ec081124f933cca6d52aab3efebc43063b61af2c3ceb3f61e47000cb/userdata/shm DeviceMajor:0 DeviceMinor:220 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b9b4add9e6248070ae9e2de7d0e34209fef2c40ab9812dd1e22ea756052e6699/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-361 DeviceMajor:0 DeviceMinor:361 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-85 DeviceMajor:0 DeviceMinor:85 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c04fe7dc-a0c3-4626-8512-a519e8821681/volumes/kubernetes.io~projected/kube-api-access-bjvgh DeviceMajor:0 DeviceMinor:211 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/910e7c90-9822-46c7-aa83-929c32dee95c/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:225 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/9c690dd2-7aa3-46c8-981b-0cc314fc2889/volumes/kubernetes.io~projected/kube-api-access-zr8pl DeviceMajor:0 DeviceMinor:234 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5e17aba9bf8cf3d0b3ce4706701e0c0bcddf8f4a9092b910c642eca0dc7e705e/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:302 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-351 DeviceMajor:0 DeviceMinor:351 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3173666e-bb4d-4f89-99bd-061c19370b91/volumes/kubernetes.io~projected/kube-api-access-67j8n DeviceMajor:0 DeviceMinor:59 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-353 DeviceMajor:0 DeviceMinor:353 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/85a01e79e10328907cc525f8fb341eefe4898bdd62ff4a885933fcd376ee6fc7/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3c3fff7263bb1bbea6d51328fa0da0d1ebce890bc0664b6d54c459b03df4ce54/userdata/shm DeviceMajor:0 DeviceMinor:347 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec/volumes/kubernetes.io~projected/kube-api-access-n69wr DeviceMajor:0 DeviceMinor:219 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/de891953be9ade7bbb17c6b8e7cee8871cb4b970879f0a38f6acd4642a740442/userdata/shm DeviceMajor:0 DeviceMinor:222 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/f77495d2-eb88-48cb-b46e-510fe055471b/volumes/kubernetes.io~projected/kube-api-access-kmfvd DeviceMajor:0 DeviceMinor:237 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/4345f2dd-427e-40d1-a5cc-43440c8cd4f0/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:45 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/bc73f5d0-671f-4cf5-8aab-325db0b149e7/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a83821ca18e0e2b64d3caa415f922b5b97cfc1c78eaf1787ff1b0e4699b130d6/userdata/shm DeviceMajor:0 DeviceMinor:231 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7d429a794704486ee79bdbc437afe85d138f2bffdcd0d2cc6eed39c04d03250c/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/29aedf8a9de47ad4db2cef7e3c96d8a34b6e7dd5e05376a6b5c2e958063947bd/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-267 DeviceMajor:0 DeviceMinor:267 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c134f615-1a89-4c6a-9540-62846db76aa0/volumes/kubernetes.io~projected/kube-api-access-gxsbg DeviceMajor:0 DeviceMinor:295 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/adb395cb5008e7e50add661ae224bf0d113a15a629ade7d908f70c8f5118805f/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4345f2dd-427e-40d1-a5cc-43440c8cd4f0/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:208 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e54831c9-77f6-4bce-b8fc-bbfd1838dddd/volumes/kubernetes.io~projected/kube-api-access-dchj2 DeviceMajor:0 DeviceMinor:236 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-280 DeviceMajor:0 DeviceMinor:280 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/28b815f8ae9435007e7952a029539da96090d444abfe27739689310371565426/userdata/shm DeviceMajor:0 DeviceMinor:326 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:178c5d7f1cec252 MacAddress:e2:4a:d2:a3:21:a3 Speed:10000 Mtu:8901} {Name:28b815f8ae94350 MacAddress:32:c1:23:b7:69:15 Speed:10000 Mtu:8901} {Name:3c3fff7263bb1bb MacAddress:8e:04:f8:a1:5d:9a Speed:10000 Mtu:8901} {Name:4beedffb5268777 MacAddress:6e:b3:cf:f2:bf:80 Speed:10000 Mtu:8901} {Name:5e17aba9bf8cf3d MacAddress:fa:27:95:7b:8d:b2 Speed:10000 Mtu:8901} {Name:5f96514100a9ea2 MacAddress:0a:de:41:3b:f8:cb Speed:10000 Mtu:8901} {Name:639b913c6602460 MacAddress:fe:eb:9c:10:62:ca Speed:10000 Mtu:8901} {Name:7d429a794704486 MacAddress:76:ab:af:77:d8:b9 Speed:10000 Mtu:8901} {Name:801d2f35ec08112 MacAddress:66:78:5f:54:9e:81 Speed:10000 Mtu:8901} {Name:85a01e79e103289 MacAddress:ce:a6:15:a4:9d:73 Speed:10000 Mtu:8901} {Name:a83821ca18e0e2b MacAddress:42:ad:20:72:c9:bd Speed:10000 Mtu:8901} {Name:b9b4add9e624807 MacAddress:16:14:b7:7b:14:69 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:fc:6b:54:b1 Speed:0 Mtu:9001} {Name:br-int MacAddress:06:3b:8c:2f:32:01 Speed:0 Mtu:8901} {Name:de891953be9ade7 MacAddress:26:b5:fd:bc:6c:8d Speed:10000 Mtu:8901} {Name:eedd95ddefbd506 MacAddress:fa:d0:3c:58:42:ff Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:fc:6b:54:b1 Speed:0 Mtu:9001} {Name:f8f31e26c79f285 MacAddress:aa:7c:13:86:5d:27 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:86:8d:5b:ae:19:eb Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:da:a0:64:ac:0a:28 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812007424 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 15:38:03.947069 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.946845 5013 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 15:38:03.947069 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.946911 5013 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 15:38:03.947220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.947177 5013 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 15:38:03.947368 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.947222 5013 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-40.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 15:38:03.947409 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.947371 5013 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 15:38:03.947409 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.947379 5013 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 15:38:03.947409 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.947389 5013 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 15:38:03.947409 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.947402 5013 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 15:38:03.947510 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.947483 5013 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 15:38:03.947638 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.947629 5013 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 15:38:03.947733 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.947725 5013 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 15:38:03.947768 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.947759 5013 kubelet.go:493] "Attempting to sync node with API server" Oct 30 15:38:03.947799 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.947771 5013 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 15:38:03.947799 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.947781 5013 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 15:38:03.947799 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.947788 5013 kubelet.go:405] "Adding apiserver pod source" Oct 30 15:38:03.947799 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.947796 5013 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 15:38:03.948493 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.948476 5013 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 15:38:03.948493 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.948490 5013 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 15:38:03.948908 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.948886 5013 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 15:38:03.951032 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951015 5013 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 15:38:03.951032 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951031 5013 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 15:38:03.951183 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951147 5013 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 30 15:38:03.951472 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951456 5013 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 15:38:03.951542 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951496 5013 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 15:38:03.951621 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951610 5013 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 15:38:03.951675 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951631 5013 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 15:38:03.951675 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951641 5013 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 15:38:03.951675 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951651 5013 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 15:38:03.951675 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951660 5013 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 15:38:03.951675 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951669 5013 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 15:38:03.951908 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951677 5013 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 15:38:03.951908 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951686 5013 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 15:38:03.951908 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951698 5013 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 15:38:03.951908 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951707 5013 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 15:38:03.951908 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951731 5013 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 15:38:03.951908 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951747 5013 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 15:38:03.951908 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951781 5013 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 15:38:03.951908 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.951791 5013 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 15:38:03.955308 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.955293 5013 server.go:1263] "Started kubelet" Oct 30 15:38:03.955430 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.955364 5013 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 15:38:03.955949 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.955610 5013 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 15:38:03.955949 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.955669 5013 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 15:38:03.955874 ip-10-0-129-40 systemd[1]: Started Kubernetes Kubelet. Oct 30 15:38:03.956179 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.956169 5013 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 15:38:03.956962 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.956945 5013 server.go:317] "Adding debug handlers to kubelet server" Oct 30 15:38:03.962316 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.962292 5013 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 15:38:03.962806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.962650 5013 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 15:38:03.962806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.962665 5013 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 15:38:03.962806 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:03.962727 5013 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-40.ec2.internal\" not found" Oct 30 15:38:03.962985 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.962923 5013 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 15:38:03.963716 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.962685 5013 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 15:38:03.966629 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.966566 5013 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 15:32:25 +0000 UTC" deadline="2028-06-01 12:50:24.390010551 +0000 UTC" Oct 30 15:38:03.966629 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.966616 5013 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13917h12m20.423398962s" Oct 30 15:38:03.967686 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.967667 5013 factory.go:55] Registering systemd factory Oct 30 15:38:03.967686 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.967687 5013 factory.go:223] Registration of the systemd container factory successfully Oct 30 15:38:03.968041 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968006 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="92898f00-aba8-475c-9389-1f4d3fae8103" volumeName="kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-image-registry-private-configuration" seLinuxMountContext="" Oct 30 15:38:03.968115 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968046 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4345f2dd-427e-40d1-a5cc-43440c8cd4f0" volumeName="kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-kube-api-access-tbkl9" seLinuxMountContext="" Oct 30 15:38:03.968115 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968059 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="521c6b52-045a-447e-9d9e-49c0de21c0a2" volumeName="kubernetes.io/projected/521c6b52-045a-447e-9d9e-49c0de21c0a2-kube-api-access-snbvb" seLinuxMountContext="" Oct 30 15:38:03.968115 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968071 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5306d4a8-24eb-461e-9384-f71ed3b41907" volumeName="kubernetes.io/configmap/5306d4a8-24eb-461e-9384-f71ed3b41907-trusted-ca" seLinuxMountContext="" Oct 30 15:38:03.968115 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968083 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5306d4a8-24eb-461e-9384-f71ed3b41907" volumeName="kubernetes.io/secret/5306d4a8-24eb-461e-9384-f71ed3b41907-serving-cert" seLinuxMountContext="" Oct 30 15:38:03.968115 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968093 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91393a38-a119-4479-adb6-f8fe6316bce3" volumeName="kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-cni-sysctl-allowlist" seLinuxMountContext="" Oct 30 15:38:03.968115 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968105 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e54831c9-77f6-4bce-b8fc-bbfd1838dddd" volumeName="kubernetes.io/configmap/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-telemetry-config" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968117 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0025b75d-4f3c-4c5a-91ca-505bda44bccb" volumeName="kubernetes.io/configmap/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-daemon-config" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968132 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4345f2dd-427e-40d1-a5cc-43440c8cd4f0" volumeName="kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-installation-pull-secrets" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968145 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="910e7c90-9822-46c7-aa83-929c32dee95c" volumeName="kubernetes.io/projected/910e7c90-9822-46c7-aa83-929c32dee95c-kube-api-access-jl229" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968170 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91393a38-a119-4479-adb6-f8fe6316bce3" volumeName="kubernetes.io/projected/91393a38-a119-4479-adb6-f8fe6316bce3-kube-api-access-vn2bz" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968186 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9eeaa1d3-abc9-4811-acad-f3a4419d967e" volumeName="kubernetes.io/configmap/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-config" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968244 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbfe6e96-808d-4fb6-bce1-306f37eaf69d" volumeName="kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968257 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb4bb521-d604-4dbe-b464-46fb320253a5" volumeName="kubernetes.io/projected/eb4bb521-d604-4dbe-b464-46fb320253a5-kube-api-access-njfdc" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968273 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e" volumeName="kubernetes.io/secret/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-signing-key" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968285 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30238695-c5e1-433b-b666-c822522e05e2" volumeName="kubernetes.io/empty-dir/30238695-c5e1-433b-b666-c822522e05e2-etc-tuned" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968298 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="92898f00-aba8-475c-9389-1f4d3fae8103" volumeName="kubernetes.io/empty-dir/92898f00-aba8-475c-9389-1f4d3fae8103-ca-trust-extracted" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968311 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bc73f5d0-671f-4cf5-8aab-325db0b149e7" volumeName="kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovnkube-script-lib" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968323 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bc73f5d0-671f-4cf5-8aab-325db0b149e7" volumeName="kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-env-overrides" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968334 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bc73f5d0-671f-4cf5-8aab-325db0b149e7" volumeName="kubernetes.io/secret/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovn-node-metrics-cert" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968345 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f77495d2-eb88-48cb-b46e-510fe055471b" volumeName="kubernetes.io/configmap/f77495d2-eb88-48cb-b46e-510fe055471b-config-volume" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968356 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f77495d2-eb88-48cb-b46e-510fe055471b" volumeName="kubernetes.io/empty-dir/f77495d2-eb88-48cb-b46e-510fe055471b-tmp-dir" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968369 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4345f2dd-427e-40d1-a5cc-43440c8cd4f0" volumeName="kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-certificates" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968384 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7322d7b1-561a-402f-a9bb-111264b04743" volumeName="kubernetes.io/configmap/7322d7b1-561a-402f-a9bb-111264b04743-nginx-conf" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968394 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eee942d8-9866-4315-ab29-d721cf68455c" volumeName="kubernetes.io/projected/eee942d8-9866-4315-ab29-d721cf68455c-kube-api-access-7twh7" seLinuxMountContext="" Oct 30 15:38:03.968418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968415 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="910e7c90-9822-46c7-aa83-929c32dee95c" volumeName="kubernetes.io/secret/910e7c90-9822-46c7-aa83-929c32dee95c-cert" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968430 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2d1df50d-be8d-4d79-ae09-2eb0853f313d" volumeName="kubernetes.io/projected/2d1df50d-be8d-4d79-ae09-2eb0853f313d-kube-api-access-z6dbr" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968444 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2d95d50f-c563-4080-909a-8ec8dc3b69b7" volumeName="kubernetes.io/empty-dir/2d95d50f-c563-4080-909a-8ec8dc3b69b7-snapshots" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968462 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4345f2dd-427e-40d1-a5cc-43440c8cd4f0" volumeName="kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-image-registry-private-configuration" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968474 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9c690dd2-7aa3-46c8-981b-0cc314fc2889" volumeName="kubernetes.io/projected/9c690dd2-7aa3-46c8-981b-0cc314fc2889-kube-api-access-zr8pl" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968493 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0025b75d-4f3c-4c5a-91ca-505bda44bccb" volumeName="kubernetes.io/projected/0025b75d-4f3c-4c5a-91ca-505bda44bccb-kube-api-access-hsp5p" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968507 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a1bfd21-ddfa-4820-8356-49214c4cfc3b" volumeName="kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-default-certificate" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968520 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23eff76f-8eed-4d55-88b9-1ae63cb1e9ec" volumeName="kubernetes.io/configmap/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-config" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968534 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2d95d50f-c563-4080-909a-8ec8dc3b69b7" volumeName="kubernetes.io/configmap/2d95d50f-c563-4080-909a-8ec8dc3b69b7-trusted-ca-bundle" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968548 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4345f2dd-427e-40d1-a5cc-43440c8cd4f0" volumeName="kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-bound-sa-token" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968562 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91393a38-a119-4479-adb6-f8fe6316bce3" volumeName="kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-cni-binary-copy" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968574 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b9bcd436-d777-4ca7-a840-bce674d247fd" volumeName="kubernetes.io/projected/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-api-access-dj24q" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968586 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1aaecc9-6c27-4adb-bf44-2c31d16fe29d" volumeName="kubernetes.io/projected/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-kube-api-access-vsnqq" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968599 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13" volumeName="kubernetes.io/projected/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-kube-api-access-zlmrz" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968611 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e" volumeName="kubernetes.io/projected/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-kube-api-access-jxv8s" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968626 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2d95d50f-c563-4080-909a-8ec8dc3b69b7" volumeName="kubernetes.io/secret/2d95d50f-c563-4080-909a-8ec8dc3b69b7-serving-cert" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968640 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30238695-c5e1-433b-b666-c822522e05e2" volumeName="kubernetes.io/empty-dir/30238695-c5e1-433b-b666-c822522e05e2-tmp" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968652 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4345f2dd-427e-40d1-a5cc-43440c8cd4f0" volumeName="kubernetes.io/empty-dir/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-ca-trust-extracted" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968665 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7df10531-4deb-4b95-ba47-9fb316775878" volumeName="kubernetes.io/configmap/7df10531-4deb-4b95-ba47-9fb316775878-konnectivity-ca" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968679 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91393a38-a119-4479-adb6-f8fe6316bce3" volumeName="kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968691 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="92898f00-aba8-475c-9389-1f4d3fae8103" volumeName="kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-trusted-ca" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968705 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e" volumeName="kubernetes.io/configmap/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-signing-cabundle" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968716 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="14a2267f-e095-45cb-b0bc-99d7ab20611a" volumeName="kubernetes.io/empty-dir/14a2267f-e095-45cb-b0bc-99d7ab20611a-tmp-dir" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968728 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3173666e-bb4d-4f89-99bd-061c19370b91" volumeName="kubernetes.io/projected/3173666e-bb4d-4f89-99bd-061c19370b91-kube-api-access-67j8n" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968739 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50c89e19-a6a1-4abe-906d-e2548c00e23a" volumeName="kubernetes.io/projected/50c89e19-a6a1-4abe-906d-e2548c00e23a-kube-api-access-72qnj" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968752 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b9bcd436-d777-4ca7-a840-bce674d247fd" volumeName="kubernetes.io/empty-dir/b9bcd436-d777-4ca7-a840-bce674d247fd-data-volume" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968763 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c04fe7dc-a0c3-4626-8512-a519e8821681" volumeName="kubernetes.io/projected/c04fe7dc-a0c3-4626-8512-a519e8821681-kube-api-access-bjvgh" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968775 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c134f615-1a89-4c6a-9540-62846db76aa0" volumeName="kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968789 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf86f01f-c44d-46a3-be13-03c99b006285" volumeName="kubernetes.io/projected/cf86f01f-c44d-46a3-be13-03c99b006285-kube-api-access-zrwsv" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968800 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ea29f558-cb33-4c42-9663-5db5361081b1" volumeName="kubernetes.io/projected/ea29f558-cb33-4c42-9663-5db5361081b1-kube-api-access-b56wc" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968812 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb4bb521-d604-4dbe-b464-46fb320253a5" volumeName="kubernetes.io/secret/eb4bb521-d604-4dbe-b464-46fb320253a5-serving-cert" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968831 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="92898f00-aba8-475c-9389-1f4d3fae8103" volumeName="kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-kube-api-access-v8xdt" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968844 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a1bfd21-ddfa-4820-8356-49214c4cfc3b" volumeName="kubernetes.io/projected/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-kube-api-access-t9dtn" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968855 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2d95d50f-c563-4080-909a-8ec8dc3b69b7" volumeName="kubernetes.io/empty-dir/2d95d50f-c563-4080-909a-8ec8dc3b69b7-tmp" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968867 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30238695-c5e1-433b-b666-c822522e05e2" volumeName="kubernetes.io/projected/30238695-c5e1-433b-b666-c822522e05e2-kube-api-access-zqmfg" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968877 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb4bb521-d604-4dbe-b464-46fb320253a5" volumeName="kubernetes.io/configmap/eb4bb521-d604-4dbe-b464-46fb320253a5-config" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968890 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0025b75d-4f3c-4c5a-91ca-505bda44bccb" volumeName="kubernetes.io/configmap/0025b75d-4f3c-4c5a-91ca-505bda44bccb-cni-binary-copy" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968901 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23eff76f-8eed-4d55-88b9-1ae63cb1e9ec" volumeName="kubernetes.io/projected/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-kube-api-access-n69wr" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968912 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2d95d50f-c563-4080-909a-8ec8dc3b69b7" volumeName="kubernetes.io/configmap/2d95d50f-c563-4080-909a-8ec8dc3b69b7-service-ca-bundle" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968923 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5306d4a8-24eb-461e-9384-f71ed3b41907" volumeName="kubernetes.io/projected/5306d4a8-24eb-461e-9384-f71ed3b41907-kube-api-access-ccndt" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968934 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="92898f00-aba8-475c-9389-1f4d3fae8103" volumeName="kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-bound-sa-token" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968946 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="92898f00-aba8-475c-9389-1f4d3fae8103" volumeName="kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-installation-pull-secrets" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968958 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9eeaa1d3-abc9-4811-acad-f3a4419d967e" volumeName="kubernetes.io/projected/9eeaa1d3-abc9-4811-acad-f3a4419d967e-kube-api-access-g95zs" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968969 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bc73f5d0-671f-4cf5-8aab-325db0b149e7" volumeName="kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovnkube-config" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968980 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a1bfd21-ddfa-4820-8356-49214c4cfc3b" volumeName="kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-stats-auth" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.968992 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23eff76f-8eed-4d55-88b9-1ae63cb1e9ec" volumeName="kubernetes.io/secret/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-serving-cert" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969004 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2d95d50f-c563-4080-909a-8ec8dc3b69b7" volumeName="kubernetes.io/projected/2d95d50f-c563-4080-909a-8ec8dc3b69b7-kube-api-access-8zdpn" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969017 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5306d4a8-24eb-461e-9384-f71ed3b41907" volumeName="kubernetes.io/configmap/5306d4a8-24eb-461e-9384-f71ed3b41907-config" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969028 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e54831c9-77f6-4bce-b8fc-bbfd1838dddd" volumeName="kubernetes.io/projected/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-kube-api-access-dchj2" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969040 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13" volumeName="kubernetes.io/configmap/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-iptables-alerter-script" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969051 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4345f2dd-427e-40d1-a5cc-43440c8cd4f0" volumeName="kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-trusted-ca" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969063 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7df10531-4deb-4b95-ba47-9fb316775878" volumeName="kubernetes.io/secret/7df10531-4deb-4b95-ba47-9fb316775878-agent-certs" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969077 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f77495d2-eb88-48cb-b46e-510fe055471b" volumeName="kubernetes.io/projected/f77495d2-eb88-48cb-b46e-510fe055471b-kube-api-access-kmfvd" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969088 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="14a2267f-e095-45cb-b0bc-99d7ab20611a" volumeName="kubernetes.io/projected/14a2267f-e095-45cb-b0bc-99d7ab20611a-kube-api-access-lvdcn" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969100 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3173666e-bb4d-4f89-99bd-061c19370b91" volumeName="kubernetes.io/configmap/3173666e-bb4d-4f89-99bd-061c19370b91-serviceca" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969111 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bc73f5d0-671f-4cf5-8aab-325db0b149e7" volumeName="kubernetes.io/projected/bc73f5d0-671f-4cf5-8aab-325db0b149e7-kube-api-access-jncfk" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969123 5013 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="92898f00-aba8-475c-9389-1f4d3fae8103" volumeName="kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-registry-certificates" seLinuxMountContext="" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969135 5013 reconstruct.go:97] "Volume reconstruction finished" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969142 5013 reconciler.go:29] "Reconciler: start to sync state" Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969430 5013 factory.go:153] Registering CRI-O factory Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969442 5013 factory.go:223] Registration of the crio container factory successfully Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969490 5013 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969508 5013 factory.go:103] Registering Raw factory Oct 30 15:38:03.969566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969522 5013 manager.go:1199] Started watching for new ooms in manager Oct 30 15:38:03.972633 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.969825 5013 manager.go:319] Starting recovery of all containers Oct 30 15:38:03.977284 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.977252 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/1.log" Oct 30 15:38:03.978333 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.978116 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/1.log" Oct 30 15:38:03.996308 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.996265 5013 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 15:38:03.997736 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.997714 5013 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 15:38:03.997736 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.997738 5013 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 15:38:03.997888 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:03.997760 5013 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 15:38:03.997888 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:03.997797 5013 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 30 15:38:04.012984 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.012936 5013 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 30 15:38:04.026895 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.026877 5013 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 15:38:04.027414 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.027393 5013 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 15:38:04.028034 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.028012 5013 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 15:38:04.028102 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.028038 5013 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 15:38:04.036634 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.036602 5013 generic.go:356] "Generic (PLEG): container finished" podID="91393a38-a119-4479-adb6-f8fe6316bce3" containerID="cb79b65197fd0ac351e62738ab215cd250806ad2fc99270111214be1d7a155a6" exitCode=0 Oct 30 15:38:04.036734 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.036638 5013 generic.go:356] "Generic (PLEG): container finished" podID="91393a38-a119-4479-adb6-f8fe6316bce3" containerID="b25fb3029d2d3e424dd30fb55471d99545f433d39828b637ac908267eb3a3dba" exitCode=0 Oct 30 15:38:04.036734 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.036650 5013 generic.go:356] "Generic (PLEG): container finished" podID="91393a38-a119-4479-adb6-f8fe6316bce3" containerID="1dd27e955f041d89def5adfde1cf27741eaf60da65862f058bb5fdc61552f2b7" exitCode=0 Oct 30 15:38:04.036734 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.036658 5013 generic.go:356] "Generic (PLEG): container finished" podID="91393a38-a119-4479-adb6-f8fe6316bce3" containerID="01ac4a76fcc6d546ea0d8a1df7046c84e43f8528081e77a46247bd117f3d709b" exitCode=0 Oct 30 15:38:04.036734 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.036666 5013 generic.go:356] "Generic (PLEG): container finished" podID="91393a38-a119-4479-adb6-f8fe6316bce3" containerID="72752f4252f81eda3690f9574decf1971cf3ab4c7def1482f08aef7ee6bf70c0" exitCode=0 Oct 30 15:38:04.036734 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.036674 5013 generic.go:356] "Generic (PLEG): container finished" podID="91393a38-a119-4479-adb6-f8fe6316bce3" containerID="c764d98a126803368212cc09e42854f7c9a31d04e2f2c5ac89333e7fd7288384" exitCode=0 Oct 30 15:38:04.041702 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.041681 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/1.log" Oct 30 15:38:04.041801 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.041717 5013 generic.go:356] "Generic (PLEG): container finished" podID="5306d4a8-24eb-461e-9384-f71ed3b41907" containerID="f0bff60ba47b0b67c8bd6bf33c99bf64c08bc4973fb67c080a9b011c64cfbfd2" exitCode=255 Oct 30 15:38:04.069383 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.069356 5013 generic.go:356] "Generic (PLEG): container finished" podID="1c39a4a8ca858e13aa8ee478d9cdc860" containerID="e670515436d377fb4ea0675a5d6cf95d4f2273b6eabfce19630feff7389fe715" exitCode=0 Oct 30 15:38:04.075011 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.074993 5013 manager.go:324] Recovery completed Oct 30 15:38:04.096484 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.096458 5013 cpu_manager.go:226] "Starting" policy="none" Oct 30 15:38:04.096484 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.096476 5013 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 15:38:04.096621 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.096496 5013 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 15:38:04.096695 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.096678 5013 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 30 15:38:04.096748 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.096697 5013 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 30 15:38:04.096748 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.096724 5013 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 30 15:38:04.096748 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.096734 5013 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 30 15:38:04.096748 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.096742 5013 policy_none.go:50] "Start" Oct 30 15:38:04.096920 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.096764 5013 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 15:38:04.096920 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.096778 5013 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 15:38:04.097486 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.097470 5013 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 30 15:38:04.097486 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.097487 5013 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 30 15:38:04.097661 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.097497 5013 policy_none.go:44] "Start" Oct 30 15:38:04.098824 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:04.098750 5013 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Oct 30 15:38:04.102997 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.102972 5013 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 15:38:04.106540 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.106526 5013 manager.go:341] "Starting Device Plugin manager" Oct 30 15:38:04.106679 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:04.106658 5013 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 15:38:04.106679 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.106677 5013 server.go:87] "Starting device plugin registration server" Oct 30 15:38:04.106967 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.106955 5013 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 15:38:04.107014 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.106969 5013 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 15:38:04.107092 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.107075 5013 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 15:38:04.107226 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.107172 5013 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 15:38:04.107279 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.107235 5013 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 15:38:04.107279 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.107242 5013 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 15:38:04.107386 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.107281 5013 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T15:38:04.10722518Z","UUID":"8978a947-c083-4b8b-817f-1b6b389273f6","Handler":null,"Name":"","Endpoint":""} Oct 30 15:38:04.108547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.108528 5013 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 15:38:04.108627 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.108555 5013 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 15:38:04.108750 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:04.108731 5013 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 15:38:04.207566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.207544 5013 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 15:38:04.208686 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.208665 5013 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientMemory" Oct 30 15:38:04.208773 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.208697 5013 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 15:38:04.208773 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.208707 5013 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-40.ec2.internal" event="NodeHasSufficientPID" Oct 30 15:38:04.208773 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.208747 5013 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.248788 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.248765 5013 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.248877 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.248816 5013 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.299394 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.299328 5013 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal"] Oct 30 15:38:04.299949 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.299933 5013 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.299997 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.299974 5013 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.299997 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.299969 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" event={"ID":"1c39a4a8ca858e13aa8ee478d9cdc860","Type":"ContainerStarted","Data":"2627bd33277369c4b1aeb1c256397639646c1d3a30cace084ee345f3191e0114"} Oct 30 15:38:04.300068 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.300018 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" event={"ID":"1c39a4a8ca858e13aa8ee478d9cdc860","Type":"ContainerDied","Data":"e670515436d377fb4ea0675a5d6cf95d4f2273b6eabfce19630feff7389fe715"} Oct 30 15:38:04.300068 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.300040 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" event={"ID":"1c39a4a8ca858e13aa8ee478d9cdc860","Type":"ContainerStarted","Data":"c13800af7a74e91e88669cd2a1bf00e88ae1c7a7292b5e83e79ab30f3df07a41"} Oct 30 15:38:04.300068 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.300057 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" event={"ID":"5ea352a0aff2f2b8b8e81204bcdb2ad9","Type":"ContainerStarted","Data":"f7241025f46123a9ef144f42d9036bd0b5ae64ba07e4dde2750f860a1ef0f24f"} Oct 30 15:38:04.300157 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.300074 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" event={"ID":"5ea352a0aff2f2b8b8e81204bcdb2ad9","Type":"ContainerStarted","Data":"edda3baa71c3486c5355cca87b5a31e02e178daa1d6cdb2a3bb497d97e64094f"} Oct 30 15:38:04.324440 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.324422 5013 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 15:38:04.324514 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:04.324455 5013 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-40.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.325134 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.325117 5013 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 15:38:04.325213 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:04.325171 5013 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.414447 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.414422 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c39a4a8ca858e13aa8ee478d9cdc860-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"1c39a4a8ca858e13aa8ee478d9cdc860\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.414548 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.414453 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5ea352a0aff2f2b8b8e81204bcdb2ad9-config\") pod \"kube-apiserver-proxy-ip-10-0-129-40.ec2.internal\" (UID: \"5ea352a0aff2f2b8b8e81204bcdb2ad9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.414548 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.414480 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c39a4a8ca858e13aa8ee478d9cdc860-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"1c39a4a8ca858e13aa8ee478d9cdc860\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.515065 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.515043 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c39a4a8ca858e13aa8ee478d9cdc860-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"1c39a4a8ca858e13aa8ee478d9cdc860\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.515142 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.515065 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c39a4a8ca858e13aa8ee478d9cdc860-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"1c39a4a8ca858e13aa8ee478d9cdc860\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.515142 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.515082 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c39a4a8ca858e13aa8ee478d9cdc860-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"1c39a4a8ca858e13aa8ee478d9cdc860\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.515142 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.515118 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5ea352a0aff2f2b8b8e81204bcdb2ad9-config\") pod \"kube-apiserver-proxy-ip-10-0-129-40.ec2.internal\" (UID: \"5ea352a0aff2f2b8b8e81204bcdb2ad9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.515292 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.515152 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5ea352a0aff2f2b8b8e81204bcdb2ad9-config\") pod \"kube-apiserver-proxy-ip-10-0-129-40.ec2.internal\" (UID: \"5ea352a0aff2f2b8b8e81204bcdb2ad9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.515292 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.515225 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c39a4a8ca858e13aa8ee478d9cdc860-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" (UID: \"1c39a4a8ca858e13aa8ee478d9cdc860\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:38:04.948931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.948902 5013 apiserver.go:52] "Watching apiserver" Oct 30 15:38:04.999765 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:04.999739 5013 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 15:38:05.000456 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.000434 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hrsgx","kube-system/global-pull-secret-syncer-kpfss","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal","openshift-image-registry/image-registry-857f96f97f-7lt52","openshift-network-diagnostics/network-check-source-6f78d59f99-nbxc9","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd","kube-system/kas-connection-checker-685895c7c9-s9vdp","openshift-ingress/router-default-779688d9fc-m8g4f","openshift-kube-storage-version-migrator/migrator-66df5d496b-jzd98","openshift-console-operator/console-operator-7cd56b8dbc-tj9gm","openshift-cluster-node-tuning-operator/tuned-vnf8v","openshift-network-console/networking-console-plugin-86c478bb88-tstct","openshift-ovn-kubernetes/ovnkube-node-v2dlp","openshift-kube-storage-version-migrator/migrator-66df5d496b-fv7tg","openshift-network-diagnostics/network-check-target-5dlc4","openshift-multus/multus-additional-cni-plugins-xwrzk","openshift-multus/multus-c4gtj","openshift-image-registry/image-registry-7ff55fd746-8jqtl","openshift-dns/node-resolver-6k2tn","kube-system/kas-connection-checker-685895c7c9-8sbdg","kube-system/konnectivity-agent-g8vnr","openshift-multus/network-metrics-daemon-c6b9v","openshift-insights/insights-operator-7c4986d449-8bj4r","openshift-service-ca/service-ca-5fffc97bdf-hgpjh","openshift-insights/insights-runtime-extractor-g47gl","openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n","openshift-network-operator/iptables-alerter-xzc7g","kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal","openshift-image-registry/node-ca-nvxqq","openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn","kube-system/kas-connection-checker-685895c7c9-cnb75","openshift-dns/dns-default-bbfpq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xcw5d"] Oct 30 15:38:05.001667 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.001649 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:38:05.002255 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.002237 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:05.002863 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.002848 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:38:05.004145 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.004126 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.004939 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.004923 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:38:05.011586 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.011567 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:38:05.013425 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.013407 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.014353 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.014337 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bbfpq" Oct 30 15:38:05.014940 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.014924 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:05.015217 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.015178 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.015305 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.015216 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.016249 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.016231 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 30 15:38:05.016346 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.016328 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.016471 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.016336 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.016471 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.016359 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.016471 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.016442 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.016594 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.016505 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 30 15:38:05.016672 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.016658 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 30 15:38:05.016705 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.016676 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 30 15:38:05.016761 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.016750 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-4vrlf\"" Oct 30 15:38:05.016838 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.016826 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 30 15:38:05.016872 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.016846 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jlnnz\"" Oct 30 15:38:05.035295 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.035272 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.035389 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.035320 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.035389 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.035339 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 30 15:38:05.035389 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.035343 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.035854 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.035838 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.035854 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.035848 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 30 15:38:05.035854 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.035853 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.036018 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.035883 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 15:38:05.036018 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.035917 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 30 15:38:05.036018 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.035926 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.036018 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.035917 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.036018 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.035942 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 15:38:05.036286 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036266 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.036389 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036320 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.036389 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036322 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.036389 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036383 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.036679 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036660 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.036755 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036692 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.036755 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036692 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.036755 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036742 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 15:38:05.036755 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036747 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.036921 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036769 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.036921 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036787 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.036921 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036813 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 15:38:05.036921 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036776 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 15:38:05.036921 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036839 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 15:38:05.036921 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036868 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.036921 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036888 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 15:38:05.036921 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036898 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.036921 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036915 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 15:38:05.036921 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036924 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 15:38:05.037302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036931 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.037302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.036988 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 30 15:38:05.037302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037047 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.037302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037081 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.037302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037125 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.037302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037135 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.037302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037168 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 30 15:38:05.037302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037177 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.037302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037178 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.037302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037253 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 30 15:38:05.037729 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037308 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 15:38:05.037729 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037352 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.037729 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037362 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 30 15:38:05.037729 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037392 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.037729 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037701 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.037729 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037711 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5dknm\"" Oct 30 15:38:05.037729 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037721 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 15:38:05.037929 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037784 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.037929 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037815 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 15:38:05.037929 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037847 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 30 15:38:05.037929 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037860 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 15:38:05.037929 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037910 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 15:38:05.037929 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.037922 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mw526\"" Oct 30 15:38:05.047636 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.047619 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 30 15:38:05.049953 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.049931 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 30 15:38:05.050374 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.050352 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 30 15:38:05.050480 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.050465 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2p9vw\"" Oct 30 15:38:05.050573 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.050553 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 30 15:38:05.050658 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.050628 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.050658 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.050642 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-84s7k\"" Oct 30 15:38:05.050787 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.050717 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 15:38:05.050837 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.050814 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-6lsxk\"" Oct 30 15:38:05.050903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.050842 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-khtmm\"" Oct 30 15:38:05.051340 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.051316 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 30 15:38:05.051417 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.051340 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-xbwc7\"" Oct 30 15:38:05.051417 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.050497 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 15:38:05.051417 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.050505 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qjh5r\"" Oct 30 15:38:05.051563 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.051428 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5wgs2\"" Oct 30 15:38:05.051563 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.051467 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qtl5k\"" Oct 30 15:38:05.051563 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.051520 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mhkxg\"" Oct 30 15:38:05.051687 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.051566 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-nlpvl\"" Oct 30 15:38:05.051742 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.051694 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 30 15:38:05.051742 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.051722 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ftvkm\"" Oct 30 15:38:05.051833 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.051746 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 30 15:38:05.051878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.051523 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 30 15:38:05.051878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.051843 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 15:38:05.052087 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.052069 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-grmxw\"" Oct 30 15:38:05.052215 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.052182 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zt274\"" Oct 30 15:38:05.052311 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.052298 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 15:38:05.052424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.052410 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9c74v\"" Oct 30 15:38:05.052551 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.052538 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 15:38:05.052611 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.052578 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 15:38:05.052672 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.052655 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fmlw7\"" Oct 30 15:38:05.052781 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.052765 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vr6wd\"" Oct 30 15:38:05.052958 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.052945 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.053016 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.052981 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 30 15:38:05.053112 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.053054 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 30 15:38:05.053112 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.051523 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ck89s\"" Oct 30 15:38:05.053112 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.053105 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rf9nq\"" Oct 30 15:38:05.053298 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.053145 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vkzl2\"" Oct 30 15:38:05.053298 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.053162 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-zwbzw\"" Oct 30 15:38:05.053298 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.053169 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xd5fx\"" Oct 30 15:38:05.053689 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.053648 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 30 15:38:05.053934 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.053910 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-pxr9g\"" Oct 30 15:38:05.054004 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.053957 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 15:38:05.054550 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.054535 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 15:38:05.054628 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.054584 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-5ljdx\"" Oct 30 15:38:05.054628 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.054601 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8vjmf\"" Oct 30 15:38:05.054723 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.054659 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 30 15:38:05.054723 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.054677 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 30 15:38:05.064801 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.064781 5013 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 15:38:05.073747 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.073638 5013 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:38:05.073747 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.073745 5013 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" Oct 30 15:38:05.097223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.097208 5013 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 15:38:05.097295 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.097247 5013 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-40.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-40.ec2.internal" Oct 30 15:38:05.102972 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.102950 5013 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 15:38:05.103042 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.102989 5013 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-40.ec2.internal" Oct 30 15:38:05.118870 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.118852 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:38:05.118936 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.118878 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/910e7c90-9822-46c7-aa83-929c32dee95c-cert\") pod \"ingress-canary-hrsgx\" (UID: \"910e7c90-9822-46c7-aa83-929c32dee95c\") " pod="openshift-ingress-canary/ingress-canary-hrsgx" Oct 30 15:38:05.118936 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.118894 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d95d50f-c563-4080-909a-8ec8dc3b69b7-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:38:05.118936 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.118919 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj24q\" (UniqueName: \"kubernetes.io/projected/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-api-access-dj24q\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:05.119037 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.118946 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-run-netns\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.119037 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.118970 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d95d50f-c563-4080-909a-8ec8dc3b69b7-serving-cert\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:38:05.119037 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.118990 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-system-cni-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.119152 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119031 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-k8s-cni-cncf-io\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.119152 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119145 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/910e7c90-9822-46c7-aa83-929c32dee95c-cert\") pod \"ingress-canary-hrsgx\" (UID: \"910e7c90-9822-46c7-aa83-929c32dee95c\") " pod="openshift-ingress-canary/ingress-canary-hrsgx" Oct 30 15:38:05.119289 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119161 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxsbg\" (UniqueName: \"kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg\") pod \"network-check-target-5dlc4\" (UID: \"c134f615-1a89-4c6a-9540-62846db76aa0\") " pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:38:05.119289 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119243 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5306d4a8-24eb-461e-9384-f71ed3b41907-trusted-ca\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:38:05.119402 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119306 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d95d50f-c563-4080-909a-8ec8dc3b69b7-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:38:05.119402 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119357 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dchj2\" (UniqueName: \"kubernetes.io/projected/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-kube-api-access-dchj2\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:38:05.119402 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119383 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-original-pull-secret\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:38:05.119514 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119418 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14a2267f-e095-45cb-b0bc-99d7ab20611a-tmp-dir\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:38:05.119514 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119496 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d95d50f-c563-4080-909a-8ec8dc3b69b7-serving-cert\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:38:05.119610 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119532 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-socket-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.119610 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119579 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5306d4a8-24eb-461e-9384-f71ed3b41907-trusted-ca\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:38:05.119610 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119580 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14a2267f-e095-45cb-b0bc-99d7ab20611a-tmp-dir\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:38:05.119610 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119604 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrwsv\" (UniqueName: \"kubernetes.io/projected/cf86f01f-c44d-46a3-be13-03c99b006285-kube-api-access-zrwsv\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:38:05.119787 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119628 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-kubernetes\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.119787 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119749 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.119787 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119777 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-registry-certificates\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.119899 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119802 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl229\" (UniqueName: \"kubernetes.io/projected/910e7c90-9822-46c7-aa83-929c32dee95c-kube-api-access-jl229\") pod \"ingress-canary-hrsgx\" (UID: \"910e7c90-9822-46c7-aa83-929c32dee95c\") " pod="openshift-ingress-canary/ingress-canary-hrsgx" Oct 30 15:38:05.119899 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119820 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-socket-dir-parent\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.119899 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119846 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:38:05.119899 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119876 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:05.120025 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119926 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.120025 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119958 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsnqq\" (UniqueName: \"kubernetes.io/projected/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-kube-api-access-vsnqq\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.120025 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.119984 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-default-certificate\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:05.120025 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120008 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-installation-pull-secrets\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.120231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120033 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.120231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120059 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovn-node-metrics-cert\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.120231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120084 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-image-registry-private-configuration\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.120231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120098 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.120231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120109 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-trusted-ca\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.120231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120134 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:38:05.120231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120146 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-registry-certificates\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.120231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120155 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.120564 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120287 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-default-certificate\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:05.120564 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120321 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:05.120564 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120352 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-image-registry-private-configuration\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.120564 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120373 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-installation-pull-secrets\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.120564 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120423 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9dtn\" (UniqueName: \"kubernetes.io/projected/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-kube-api-access-t9dtn\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:05.120564 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120463 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-cni-bin\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.120564 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120471 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.120564 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120505 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-trusted-ca\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.120564 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120536 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:38:05.120809 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120574 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d95d50f-c563-4080-909a-8ec8dc3b69b7-service-ca-bundle\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:38:05.120809 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120622 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-iptables-alerter-script\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:38:05.120809 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120652 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snbvb\" (UniqueName: \"kubernetes.io/projected/521c6b52-045a-447e-9d9e-49c0de21c0a2-kube-api-access-snbvb\") pod \"migrator-66df5d496b-fv7tg\" (UID: \"521c6b52-045a-447e-9d9e-49c0de21c0a2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fv7tg" Oct 30 15:38:05.120809 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120626 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovn-node-metrics-cert\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.120809 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120678 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysctl-d\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.120809 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120736 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d95d50f-c563-4080-909a-8ec8dc3b69b7-service-ca-bundle\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:38:05.120809 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120766 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-var-lib-kubelet\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.120809 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120787 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-hostroot\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.120809 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120803 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-sys-fs\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.121082 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120819 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-systemd-units\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.121082 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120836 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqmfg\" (UniqueName: \"kubernetes.io/projected/30238695-c5e1-433b-b666-c822522e05e2-kube-api-access-zqmfg\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.121082 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120852 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-iptables-alerter-script\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:38:05.121082 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120869 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72qnj\" (UniqueName: \"kubernetes.io/projected/50c89e19-a6a1-4abe-906d-e2548c00e23a-kube-api-access-72qnj\") pod \"migrator-66df5d496b-jzd98\" (UID: \"50c89e19-a6a1-4abe-906d-e2548c00e23a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-jzd98" Oct 30 15:38:05.121082 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120901 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-kubelet\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.121082 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120940 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92898f00-aba8-475c-9389-1f4d3fae8103-ca-trust-extracted\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.121082 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120976 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovnkube-script-lib\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.121082 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121000 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysconfig\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.121082 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.120999 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92898f00-aba8-475c-9389-1f4d3fae8103-ca-trust-extracted\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.121424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121087 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-run\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.121424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121140 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7322d7b1-561a-402f-a9bb-111264b04743-nginx-conf\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:38:05.121424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121160 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-host-slash\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:38:05.121424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121172 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovnkube-script-lib\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.121424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121175 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b9bcd436-d777-4ca7-a840-bce674d247fd-crio-socket\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:05.121424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121290 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-system-cni-dir\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.121424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121333 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8xdt\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-kube-api-access-v8xdt\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.121424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121344 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7322d7b1-561a-402f-a9bb-111264b04743-nginx-conf\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:38:05.121424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121353 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-systemd\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.121424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121377 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jncfk\" (UniqueName: \"kubernetes.io/projected/bc73f5d0-671f-4cf5-8aab-325db0b149e7-kube-api-access-jncfk\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.121424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121402 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zdpn\" (UniqueName: \"kubernetes.io/projected/2d95d50f-c563-4080-909a-8ec8dc3b69b7-kube-api-access-8zdpn\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:38:05.121424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121419 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121440 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-var-lib-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121459 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6dbr\" (UniqueName: \"kubernetes.io/projected/2d1df50d-be8d-4d79-ae09-2eb0853f313d-kube-api-access-z6dbr\") pod \"kas-connection-checker-685895c7c9-s9vdp\" (UID: \"2d1df50d-be8d-4d79-ae09-2eb0853f313d\") " pod="kube-system/kas-connection-checker-685895c7c9-s9vdp" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121475 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr8pl\" (UniqueName: \"kubernetes.io/projected/9c690dd2-7aa3-46c8-981b-0cc314fc2889-kube-api-access-zr8pl\") pod \"network-check-source-6f78d59f99-nbxc9\" (UID: \"9c690dd2-7aa3-46c8-981b-0cc314fc2889\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nbxc9" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121521 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121554 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-etc-kubernetes\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121642 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-signing-cabundle\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121698 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eb4bb521-d604-4dbe-b464-46fb320253a5-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121745 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-cni-netd\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121777 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-modprobe-d\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121809 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d95d50f-c563-4080-909a-8ec8dc3b69b7-tmp\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121830 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-signing-cabundle\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121843 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eb4bb521-d604-4dbe-b464-46fb320253a5-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121864 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5306d4a8-24eb-461e-9384-f71ed3b41907-config\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121886 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-stats-auth\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121903 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjvgh\" (UniqueName: \"kubernetes.io/projected/c04fe7dc-a0c3-4626-8512-a519e8821681-kube-api-access-bjvgh\") pod \"kas-connection-checker-685895c7c9-8sbdg\" (UID: \"c04fe7dc-a0c3-4626-8512-a519e8821681\") " pod="kube-system/kas-connection-checker-685895c7c9-8sbdg" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121933 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67j8n\" (UniqueName: \"kubernetes.io/projected/3173666e-bb4d-4f89-99bd-061c19370b91-kube-api-access-67j8n\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:38:05.121959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121951 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d95d50f-c563-4080-909a-8ec8dc3b69b7-tmp\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.121954 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-sys\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122017 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-host\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122047 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-serving-cert\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122074 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvdcn\" (UniqueName: \"kubernetes.io/projected/14a2267f-e095-45cb-b0bc-99d7ab20611a-kube-api-access-lvdcn\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122080 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5306d4a8-24eb-461e-9384-f71ed3b41907-config\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122102 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-stats-auth\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122233 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122250 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-serving-cert\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122268 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3173666e-bb4d-4f89-99bd-061c19370b91-serviceca\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122308 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122361 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2d95d50f-c563-4080-909a-8ec8dc3b69b7-snapshots\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122399 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122449 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-multus-certs\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122475 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2d95d50f-c563-4080-909a-8ec8dc3b69b7-snapshots\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122492 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5306d4a8-24eb-461e-9384-f71ed3b41907-serving-cert\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122525 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/14a2267f-e095-45cb-b0bc-99d7ab20611a-hosts-file\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122526 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3173666e-bb4d-4f89-99bd-061c19370b91-serviceca\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122567 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-cnibin\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122594 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eb4bb521-d604-4dbe-b464-46fb320253a5-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122620 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-device-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122644 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:05.122712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122668 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5306d4a8-24eb-461e-9384-f71ed3b41907-serving-cert\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122742 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-netns\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122755 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eb4bb521-d604-4dbe-b464-46fb320253a5-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122764 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-cni-multus\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122790 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-systemd\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122819 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-etc-selinux\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122847 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-dbus\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122870 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f77495d2-eb88-48cb-b46e-510fe055471b-config-volume\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122906 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30238695-c5e1-433b-b666-c822522e05e2-tmp\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122945 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlmrz\" (UniqueName: \"kubernetes.io/projected/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-kube-api-access-zlmrz\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.122963 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3173666e-bb4d-4f89-99bd-061c19370b91-host\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123009 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-config\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123045 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30238695-c5e1-433b-b666-c822522e05e2-tmp\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123076 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f77495d2-eb88-48cb-b46e-510fe055471b-config-volume\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123080 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-cni-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123112 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-signing-key\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123179 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-config\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123214 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-ca-trust-extracted\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123230 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbkl9\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-kube-api-access-tbkl9\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123247 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-kubelet\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123263 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-env-overrides\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123278 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmfvd\" (UniqueName: \"kubernetes.io/projected/f77495d2-eb88-48cb-b46e-510fe055471b-kube-api-access-kmfvd\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123289 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-signing-key\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123328 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g95zs\" (UniqueName: \"kubernetes.io/projected/9eeaa1d3-abc9-4811-acad-f3a4419d967e-kube-api-access-g95zs\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123357 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-ca-trust-extracted\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.123378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123358 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n69wr\" (UniqueName: \"kubernetes.io/projected/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-kube-api-access-n69wr\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123414 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-cnibin\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123427 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-env-overrides\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123440 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9bcd436-d777-4ca7-a840-bce674d247fd-data-volume\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123492 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7twh7\" (UniqueName: \"kubernetes.io/projected/eee942d8-9866-4315-ab29-d721cf68455c-kube-api-access-7twh7\") pod \"kas-connection-checker-685895c7c9-cnb75\" (UID: \"eee942d8-9866-4315-ab29-d721cf68455c\") " pod="kube-system/kas-connection-checker-685895c7c9-cnb75" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123538 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-image-registry-private-configuration\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123567 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-node-log\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123572 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9bcd436-d777-4ca7-a840-bce674d247fd-data-volume\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123592 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0025b75d-4f3c-4c5a-91ca-505bda44bccb-cni-binary-copy\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123635 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-conf-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123670 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123731 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysctl-conf\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123744 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-image-registry-private-configuration\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123808 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b56wc\" (UniqueName: \"kubernetes.io/projected/ea29f558-cb33-4c42-9663-5db5361081b1-kube-api-access-b56wc\") pod \"volume-data-source-validator-58c9f5f46d-xcw5d\" (UID: \"ea29f558-cb33-4c42-9663-5db5361081b1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xcw5d" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123838 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccndt\" (UniqueName: \"kubernetes.io/projected/5306d4a8-24eb-461e-9384-f71ed3b41907-kube-api-access-ccndt\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123864 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0025b75d-4f3c-4c5a-91ca-505bda44bccb-cni-binary-copy\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123867 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123904 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123929 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-lib-modules\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.123961 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-cni-bin\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124003 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-daemon-config\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124042 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-trusted-ca\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124061 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-slash\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124076 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-etc-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124092 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77495d2-eb88-48cb-b46e-510fe055471b-tmp-dir\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124110 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsp5p\" (UniqueName: \"kubernetes.io/projected/0025b75d-4f3c-4c5a-91ca-505bda44bccb-kube-api-access-hsp5p\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124132 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-bound-sa-token\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124148 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn2bz\" (UniqueName: \"kubernetes.io/projected/91393a38-a119-4479-adb6-f8fe6316bce3-kube-api-access-vn2bz\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124158 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77495d2-eb88-48cb-b46e-510fe055471b-tmp-dir\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124163 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-registration-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124181 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/30238695-c5e1-433b-b666-c822522e05e2-etc-tuned\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124187 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-daemon-config\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124219 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovnkube-config\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124249 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-kubelet-config\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124280 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124281 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/30238695-c5e1-433b-b666-c822522e05e2-etc-tuned\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124326 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124392 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7df10531-4deb-4b95-ba47-9fb316775878-agent-certs\") pod \"konnectivity-agent-g8vnr\" (UID: \"7df10531-4deb-4b95-ba47-9fb316775878\") " pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124418 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc73f5d0-671f-4cf5-8aab-325db0b149e7-ovnkube-config\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124475 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxv8s\" (UniqueName: \"kubernetes.io/projected/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-kube-api-access-jxv8s\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124501 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91393a38-a119-4479-adb6-f8fe6316bce3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124529 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njfdc\" (UniqueName: \"kubernetes.io/projected/eb4bb521-d604-4dbe-b464-46fb320253a5-kube-api-access-njfdc\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124532 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:38:05.124547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124570 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-bound-sa-token\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.125796 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124626 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.125796 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124642 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-ovn\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.125796 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124657 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-log-socket\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.125796 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124675 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:38:05.125796 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124693 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-certificates\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.125796 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124724 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-installation-pull-secrets\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.125796 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124744 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7df10531-4deb-4b95-ba47-9fb316775878-agent-certs\") pod \"konnectivity-agent-g8vnr\" (UID: \"7df10531-4deb-4b95-ba47-9fb316775878\") " pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:38:05.125796 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124754 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7df10531-4deb-4b95-ba47-9fb316775878-konnectivity-ca\") pod \"konnectivity-agent-g8vnr\" (UID: \"7df10531-4deb-4b95-ba47-9fb316775878\") " pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:38:05.125796 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124812 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-trusted-ca\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.125796 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124835 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-certificates\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.125796 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124965 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7df10531-4deb-4b95-ba47-9fb316775878-konnectivity-ca\") pod \"konnectivity-agent-g8vnr\" (UID: \"7df10531-4deb-4b95-ba47-9fb316775878\") " pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:38:05.125796 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.124968 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-installation-pull-secrets\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.142785 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.142758 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj24q\" (UniqueName: \"kubernetes.io/projected/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-api-access-dj24q\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:05.144794 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.144778 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxsbg\" (UniqueName: \"kubernetes.io/projected/c134f615-1a89-4c6a-9540-62846db76aa0-kube-api-access-gxsbg\") pod \"network-check-target-5dlc4\" (UID: \"c134f615-1a89-4c6a-9540-62846db76aa0\") " pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:38:05.148472 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.148452 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsnqq\" (UniqueName: \"kubernetes.io/projected/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-kube-api-access-vsnqq\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.149803 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.149786 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dchj2\" (UniqueName: \"kubernetes.io/projected/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-kube-api-access-dchj2\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:38:05.150650 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.150631 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8xdt\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-kube-api-access-v8xdt\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.150714 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.150687 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl229\" (UniqueName: \"kubernetes.io/projected/910e7c90-9822-46c7-aa83-929c32dee95c-kube-api-access-jl229\") pod \"ingress-canary-hrsgx\" (UID: \"910e7c90-9822-46c7-aa83-929c32dee95c\") " pod="openshift-ingress-canary/ingress-canary-hrsgx" Oct 30 15:38:05.151379 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.151364 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9dtn\" (UniqueName: \"kubernetes.io/projected/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-kube-api-access-t9dtn\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:05.157541 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.157525 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72qnj\" (UniqueName: \"kubernetes.io/projected/50c89e19-a6a1-4abe-906d-e2548c00e23a-kube-api-access-72qnj\") pod \"migrator-66df5d496b-jzd98\" (UID: \"50c89e19-a6a1-4abe-906d-e2548c00e23a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-jzd98" Oct 30 15:38:05.157669 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.157655 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n69wr\" (UniqueName: \"kubernetes.io/projected/23eff76f-8eed-4d55-88b9-1ae63cb1e9ec-kube-api-access-n69wr\") pod \"service-ca-operator-544d749c8c-ksnkn\" (UID: \"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" Oct 30 15:38:05.162637 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.162617 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvdcn\" (UniqueName: \"kubernetes.io/projected/14a2267f-e095-45cb-b0bc-99d7ab20611a-kube-api-access-lvdcn\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:38:05.163129 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.163092 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr8pl\" (UniqueName: \"kubernetes.io/projected/9c690dd2-7aa3-46c8-981b-0cc314fc2889-kube-api-access-zr8pl\") pod \"network-check-source-6f78d59f99-nbxc9\" (UID: \"9c690dd2-7aa3-46c8-981b-0cc314fc2889\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nbxc9" Oct 30 15:38:05.164551 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.164535 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b56wc\" (UniqueName: \"kubernetes.io/projected/ea29f558-cb33-4c42-9663-5db5361081b1-kube-api-access-b56wc\") pod \"volume-data-source-validator-58c9f5f46d-xcw5d\" (UID: \"ea29f558-cb33-4c42-9663-5db5361081b1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xcw5d" Oct 30 15:38:05.164789 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.164774 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67j8n\" (UniqueName: \"kubernetes.io/projected/3173666e-bb4d-4f89-99bd-061c19370b91-kube-api-access-67j8n\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:38:05.165858 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.165841 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccndt\" (UniqueName: \"kubernetes.io/projected/5306d4a8-24eb-461e-9384-f71ed3b41907-kube-api-access-ccndt\") pod \"console-operator-7cd56b8dbc-tj9gm\" (UID: \"5306d4a8-24eb-461e-9384-f71ed3b41907\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:38:05.167497 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.167481 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g95zs\" (UniqueName: \"kubernetes.io/projected/9eeaa1d3-abc9-4811-acad-f3a4419d967e-kube-api-access-g95zs\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:38:05.168247 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.168230 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jncfk\" (UniqueName: \"kubernetes.io/projected/bc73f5d0-671f-4cf5-8aab-325db0b149e7-kube-api-access-jncfk\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.168315 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.168255 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjvgh\" (UniqueName: \"kubernetes.io/projected/c04fe7dc-a0c3-4626-8512-a519e8821681-kube-api-access-bjvgh\") pod \"kas-connection-checker-685895c7c9-8sbdg\" (UID: \"c04fe7dc-a0c3-4626-8512-a519e8821681\") " pod="kube-system/kas-connection-checker-685895c7c9-8sbdg" Oct 30 15:38:05.170476 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.170453 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-bound-sa-token\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.173362 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.173341 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxv8s\" (UniqueName: \"kubernetes.io/projected/2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e-kube-api-access-jxv8s\") pod \"service-ca-5fffc97bdf-hgpjh\" (UID: \"2cd4f53f-cc43-43da-bde1-c9e8d5a23e6e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-hgpjh" Oct 30 15:38:05.173485 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.173470 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njfdc\" (UniqueName: \"kubernetes.io/projected/eb4bb521-d604-4dbe-b464-46fb320253a5-kube-api-access-njfdc\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr\" (UID: \"eb4bb521-d604-4dbe-b464-46fb320253a5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" Oct 30 15:38:05.174188 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.174174 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmfvd\" (UniqueName: \"kubernetes.io/projected/f77495d2-eb88-48cb-b46e-510fe055471b-kube-api-access-kmfvd\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:38:05.174348 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.174330 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7twh7\" (UniqueName: \"kubernetes.io/projected/eee942d8-9866-4315-ab29-d721cf68455c-kube-api-access-7twh7\") pod \"kas-connection-checker-685895c7c9-cnb75\" (UID: \"eee942d8-9866-4315-ab29-d721cf68455c\") " pod="kube-system/kas-connection-checker-685895c7c9-cnb75" Oct 30 15:38:05.174380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.174347 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn2bz\" (UniqueName: \"kubernetes.io/projected/91393a38-a119-4479-adb6-f8fe6316bce3-kube-api-access-vn2bz\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.175994 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.175973 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zdpn\" (UniqueName: \"kubernetes.io/projected/2d95d50f-c563-4080-909a-8ec8dc3b69b7-kube-api-access-8zdpn\") pod \"insights-operator-7c4986d449-8bj4r\" (UID: \"2d95d50f-c563-4080-909a-8ec8dc3b69b7\") " pod="openshift-insights/insights-operator-7c4986d449-8bj4r" Oct 30 15:38:05.176129 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.176115 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6dbr\" (UniqueName: \"kubernetes.io/projected/2d1df50d-be8d-4d79-ae09-2eb0853f313d-kube-api-access-z6dbr\") pod \"kas-connection-checker-685895c7c9-s9vdp\" (UID: \"2d1df50d-be8d-4d79-ae09-2eb0853f313d\") " pod="kube-system/kas-connection-checker-685895c7c9-s9vdp" Oct 30 15:38:05.177600 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.177582 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snbvb\" (UniqueName: \"kubernetes.io/projected/521c6b52-045a-447e-9d9e-49c0de21c0a2-kube-api-access-snbvb\") pod \"migrator-66df5d496b-fv7tg\" (UID: \"521c6b52-045a-447e-9d9e-49c0de21c0a2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fv7tg" Oct 30 15:38:05.177883 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.177865 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlmrz\" (UniqueName: \"kubernetes.io/projected/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-kube-api-access-zlmrz\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:38:05.177918 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.177883 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-bound-sa-token\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.178942 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.178913 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrwsv\" (UniqueName: \"kubernetes.io/projected/cf86f01f-c44d-46a3-be13-03c99b006285-kube-api-access-zrwsv\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:38:05.178942 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.178937 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsp5p\" (UniqueName: \"kubernetes.io/projected/0025b75d-4f3c-4c5a-91ca-505bda44bccb-kube-api-access-hsp5p\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.179386 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.179370 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqmfg\" (UniqueName: \"kubernetes.io/projected/30238695-c5e1-433b-b666-c822522e05e2-kube-api-access-zqmfg\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.179841 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.179826 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbkl9\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-kube-api-access-tbkl9\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.225360 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225299 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-cnibin\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.225360 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225334 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-device-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.225360 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225355 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:05.225587 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225380 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-netns\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.225587 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225403 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-cni-multus\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.225587 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225410 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-cnibin\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.225587 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225425 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-systemd\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.225587 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225446 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-device-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.225587 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225477 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-systemd\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.225587 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225471 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-netns\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.225587 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225486 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-cni-multus\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.225587 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225515 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-etc-selinux\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.225587 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225559 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-dbus\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:38:05.225587 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225590 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-etc-selinux\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.225964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225589 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3173666e-bb4d-4f89-99bd-061c19370b91-host\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:38:05.225964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225618 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-cni-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.225964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225624 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3173666e-bb4d-4f89-99bd-061c19370b91-host\") pod \"node-ca-nvxqq\" (UID: \"3173666e-bb4d-4f89-99bd-061c19370b91\") " pod="openshift-image-registry/node-ca-nvxqq" Oct 30 15:38:05.225964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225658 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-kubelet\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.225964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225670 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-dbus\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:38:05.225964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225677 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-cnibin\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.225964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225704 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-cni-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.225964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225714 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-cnibin\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.225964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225739 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-kubelet\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.225964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225811 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-node-log\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.225964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.225956 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-service-ca-bundle\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:05.226316 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226290 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-node-log\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.226398 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226330 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-conf-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.226398 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226346 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.226398 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226361 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysctl-conf\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.226398 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226381 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.226398 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226396 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226412 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-lib-modules\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226415 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226435 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-cni-bin\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226440 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-conf-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226459 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-slash\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226481 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-etc-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226486 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-cni-bin\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226499 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226505 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-registration-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226530 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-kubelet-config\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226544 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-slash\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226547 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysctl-conf\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226550 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-etc-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226560 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-lib-modules\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226557 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226589 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bbfe6e96-808d-4fb6-bce1-306f37eaf69d-kubelet-config\") pod \"global-pull-secret-syncer-kpfss\" (UID: \"bbfe6e96-808d-4fb6-bce1-306f37eaf69d\") " pod="kube-system/global-pull-secret-syncer-kpfss" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226594 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226597 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-registration-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226668 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-ovn\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226693 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-log-socket\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226707 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-ovn\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226713 5013 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226758 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-log-socket\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226758 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226809 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-run-netns\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226835 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-system-cni-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.226856 5013 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226865 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-k8s-cni-cncf-io\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226896 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-socket-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226922 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-system-cni-dir\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.226931 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls podName:e54831c9-77f6-4bce-b8fc-bbfd1838dddd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:05.726912404 +0000 UTC m=+1.877809432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-xqb8n" (UID: "e54831c9-77f6-4bce-b8fc-bbfd1838dddd") : secret "cluster-monitoring-operator-tls" not found Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226953 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-kubernetes\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226960 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-socket-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226983 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-socket-dir-parent\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226990 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-k8s-cni-cncf-io\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227003 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-kubernetes\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.226857 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-run-netns\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227008 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227062 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-multus-socket-dir-parent\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227073 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227103 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227129 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227152 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-cni-bin\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227161 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227172 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227213 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysctl-d\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227234 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-var-lib-kubelet\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.227242 5013 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227252 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-hostroot\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227273 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-sys-fs\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.227290 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs podName:0a1bfd21-ddfa-4820-8356-49214c4cfc3b nodeName:}" failed. No retries permitted until 2026-10-30 15:38:05.7272738 +0000 UTC m=+1.878170809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs") pod "router-default-779688d9fc-m8g4f" (UID: "0a1bfd21-ddfa-4820-8356-49214c4cfc3b") : secret "router-metrics-certs-default" not found Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227295 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227322 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-systemd-units\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227324 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1aaecc9-6c27-4adb-bf44-2c31d16fe29d-sys-fs\") pod \"aws-ebs-csi-driver-node-z7qm4\" (UID: \"c1aaecc9-6c27-4adb-bf44-2c31d16fe29d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7qm4" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227341 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-kubelet\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227361 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysconfig\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227372 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-var-lib-kubelet\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227380 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-run\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227410 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-run\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227409 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-host-slash\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227417 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysctl-d\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227456 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-cni-bin\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227458 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-systemd-units\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227480 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-hostroot\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227508 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9bcd436-d777-4ca7-a840-bce674d247fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227530 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-sysconfig\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227529 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13-host-slash\") pod \"iptables-alerter-xzc7g\" (UID: \"2108a090-3b2a-4b2e-a0d2-c0fe3a2ade13\") " pod="openshift-network-operator/iptables-alerter-xzc7g" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227554 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b9bcd436-d777-4ca7-a840-bce674d247fd-crio-socket\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227557 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-var-lib-kubelet\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227583 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b9bcd436-d777-4ca7-a840-bce674d247fd-crio-socket\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227591 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-system-cni-dir\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227626 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-systemd\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227633 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91393a38-a119-4479-adb6-f8fe6316bce3-system-cni-dir\") pod \"multus-additional-cni-plugins-xwrzk\" (UID: \"91393a38-a119-4479-adb6-f8fe6316bce3\") " pod="openshift-multus/multus-additional-cni-plugins-xwrzk" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227658 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227668 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-run-systemd\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227689 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-var-lib-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227734 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-var-lib-openvswitch\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.227745 5013 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227787 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.227797 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls podName:b9bcd436-d777-4ca7-a840-bce674d247fd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:05.727783202 +0000 UTC m=+1.878680226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g47gl" (UID: "b9bcd436-d777-4ca7-a840-bce674d247fd") : secret "insights-runtime-extractor-tls" not found Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227824 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-etc-kubernetes\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227854 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-cni-netd\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227879 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-modprobe-d\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227908 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-sys\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227927 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-host\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227948 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227985 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227992 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-host\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227996 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-etc-kubernetes\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.228005 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.227946 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc73f5d0-671f-4cf5-8aab-325db0b149e7-host-cni-netd\") pod \"ovnkube-node-v2dlp\" (UID: \"bc73f5d0-671f-4cf5-8aab-325db0b149e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.228023 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-multus-certs\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.228041 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-etc-modprobe-d\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.228090 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0025b75d-4f3c-4c5a-91ca-505bda44bccb-host-run-multus-certs\") pod \"multus-c4gtj\" (UID: \"0025b75d-4f3c-4c5a-91ca-505bda44bccb\") " pod="openshift-multus/multus-c4gtj" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.228046 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/14a2267f-e095-45cb-b0bc-99d7ab20611a-hosts-file\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.228113 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/14a2267f-e095-45cb-b0bc-99d7ab20611a-hosts-file\") pod \"node-resolver-6k2tn\" (UID: \"14a2267f-e095-45cb-b0bc-99d7ab20611a\") " pod="openshift-dns/node-resolver-6k2tn" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.228045 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30238695-c5e1-433b-b666-c822522e05e2-sys\") pod \"tuned-vnf8v\" (UID: \"30238695-c5e1-433b-b666-c822522e05e2\") " pod="openshift-cluster-node-tuning-operator/tuned-vnf8v" Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.228171 5013 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 15:38:05.228449 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.228226 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert podName:7322d7b1-561a-402f-a9bb-111264b04743 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:05.728212752 +0000 UTC m=+1.879109778 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-tstct" (UID: "7322d7b1-561a-402f-a9bb-111264b04743") : secret "networking-console-plugin-cert" not found Oct 30 15:38:05.233091 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.230845 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls\") pod \"image-registry-7ff55fd746-8jqtl\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.233091 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.231168 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf86f01f-c44d-46a3-be13-03c99b006285-metrics-certs\") pod \"network-metrics-daemon-c6b9v\" (UID: \"cf86f01f-c44d-46a3-be13-03c99b006285\") " pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:38:05.233091 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.231421 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls\") pod \"image-registry-857f96f97f-7lt52\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.233091 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.231434 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77495d2-eb88-48cb-b46e-510fe055471b-metrics-tls\") pod \"dns-default-bbfpq\" (UID: \"f77495d2-eb88-48cb-b46e-510fe055471b\") " pod="openshift-dns/dns-default-bbfpq" Oct 30 15:38:05.233091 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.231951 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9eeaa1d3-abc9-4811-acad-f3a4419d967e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-w2kcd\" (UID: \"9eeaa1d3-abc9-4811-acad-f3a4419d967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:38:05.301909 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.301884 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" Oct 30 15:38:05.302931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.302909 5013 scope.go:122] "RemoveContainer" containerID="f0bff60ba47b0b67c8bd6bf33c99bf64c08bc4973fb67c080a9b011c64cfbfd2" Oct 30 15:38:05.305149 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.305130 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:05.312368 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.312348 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c6b9v" Oct 30 15:38:05.314669 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.314654 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bbfpq" Oct 30 15:38:05.314819 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.314803 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:05.350673 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.350649 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:05.630147 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.630123 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd"] Oct 30 15:38:05.662359 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.662341 5013 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 15:38:05.670406 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.670386 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-857f96f97f-7lt52"] Oct 30 15:38:05.671694 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:05.671671 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92898f00_aba8_475c_9389_1f4d3fae8103.slice/crio-b43483199697c2ba06e9a2848696f842b28011c5d0a35c572cf0ea76ee3e6f7b WatchSource:0}: Error finding container b43483199697c2ba06e9a2848696f842b28011c5d0a35c572cf0ea76ee3e6f7b: Status 404 returned error can't find the container with id b43483199697c2ba06e9a2848696f842b28011c5d0a35c572cf0ea76ee3e6f7b Oct 30 15:38:05.688780 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.688764 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7ff55fd746-8jqtl"] Oct 30 15:38:05.698274 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:05.698250 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4345f2dd_427e_40d1_a5cc_43440c8cd4f0.slice/crio-cc074f279c00686cef738b2cd78f5f5d6782b28b4839cc4024d16a4112eff5f6 WatchSource:0}: Error finding container cc074f279c00686cef738b2cd78f5f5d6782b28b4839cc4024d16a4112eff5f6: Status 404 returned error can't find the container with id cc074f279c00686cef738b2cd78f5f5d6782b28b4839cc4024d16a4112eff5f6 Oct 30 15:38:05.734105 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.734080 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:38:05.734228 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.734140 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:05.734317 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.734182 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:05.734377 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.734270 5013 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 15:38:05.734377 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.734366 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:38:05.734470 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.734417 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls podName:e54831c9-77f6-4bce-b8fc-bbfd1838dddd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:06.73439807 +0000 UTC m=+2.885295086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-xqb8n" (UID: "e54831c9-77f6-4bce-b8fc-bbfd1838dddd") : secret "cluster-monitoring-operator-tls" not found Oct 30 15:38:05.734470 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.734347 5013 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 15:38:05.734560 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.734470 5013 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 15:38:05.734560 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.734488 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs podName:0a1bfd21-ddfa-4820-8356-49214c4cfc3b nodeName:}" failed. No retries permitted until 2026-10-30 15:38:06.734472163 +0000 UTC m=+2.885369195 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs") pod "router-default-779688d9fc-m8g4f" (UID: "0a1bfd21-ddfa-4820-8356-49214c4cfc3b") : secret "router-metrics-certs-default" not found Oct 30 15:38:05.734560 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.734348 5013 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 15:38:05.734560 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.734517 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert podName:7322d7b1-561a-402f-a9bb-111264b04743 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:06.734502629 +0000 UTC m=+2.885399655 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-tstct" (UID: "7322d7b1-561a-402f-a9bb-111264b04743") : secret "networking-console-plugin-cert" not found Oct 30 15:38:05.734560 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:05.734533 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls podName:b9bcd436-d777-4ca7-a840-bce674d247fd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:06.734523183 +0000 UTC m=+2.885420229 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g47gl" (UID: "b9bcd436-d777-4ca7-a840-bce674d247fd") : secret "insights-runtime-extractor-tls" not found Oct 30 15:38:05.740251 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.740234 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c6b9v"] Oct 30 15:38:05.741418 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:05.741395 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf86f01f_c44d_46a3_be13_03c99b006285.slice/crio-1a6f89187a422b530424b91219a9c401fd08e518e8c156278e041de2ce5a0421 WatchSource:0}: Error finding container 1a6f89187a422b530424b91219a9c401fd08e518e8c156278e041de2ce5a0421: Status 404 returned error can't find the container with id 1a6f89187a422b530424b91219a9c401fd08e518e8c156278e041de2ce5a0421 Oct 30 15:38:05.745554 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.745510 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:05.748662 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.748641 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bbfpq"] Oct 30 15:38:05.751137 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:05.751118 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf77495d2_eb88_48cb_b46e_510fe055471b.slice/crio-e9a1728779b816ae161b8fb84c888ec60504ba22b88c202f9f10e289d5197ccd WatchSource:0}: Error finding container e9a1728779b816ae161b8fb84c888ec60504ba22b88c202f9f10e289d5197ccd: Status 404 returned error can't find the container with id e9a1728779b816ae161b8fb84c888ec60504ba22b88c202f9f10e289d5197ccd Oct 30 15:38:05.941985 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.941955 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:05.971250 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:05.971217 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6k2tn_14a2267f-e095-45cb-b0bc-99d7ab20611a/dns-node-resolver/0.log" Oct 30 15:38:06.078684 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.078598 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-857f96f97f-7lt52" event={"ID":"92898f00-aba8-475c-9389-1f4d3fae8103","Type":"ContainerStarted","Data":"e9d722e2feb74e14470826b9c40e43b76032adb41715a42adbd22e31840ab5a2"} Oct 30 15:38:06.078684 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.078653 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-857f96f97f-7lt52" event={"ID":"92898f00-aba8-475c-9389-1f4d3fae8103","Type":"ContainerStarted","Data":"b43483199697c2ba06e9a2848696f842b28011c5d0a35c572cf0ea76ee3e6f7b"} Oct 30 15:38:06.079910 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.079886 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bbfpq" event={"ID":"f77495d2-eb88-48cb-b46e-510fe055471b","Type":"ContainerStarted","Data":"e9a1728779b816ae161b8fb84c888ec60504ba22b88c202f9f10e289d5197ccd"} Oct 30 15:38:06.081061 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.081039 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" event={"ID":"9eeaa1d3-abc9-4811-acad-f3a4419d967e","Type":"ContainerStarted","Data":"faa3f5a2645a6932f247619f412e0fd68187441d03ba98ee1dac2c29226bb9cd"} Oct 30 15:38:06.082243 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.082220 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c6b9v" event={"ID":"cf86f01f-c44d-46a3-be13-03c99b006285","Type":"ContainerStarted","Data":"1a6f89187a422b530424b91219a9c401fd08e518e8c156278e041de2ce5a0421"} Oct 30 15:38:06.083609 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.083584 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 15:38:06.083977 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.083962 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/1.log" Oct 30 15:38:06.084041 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.083991 5013 generic.go:356] "Generic (PLEG): container finished" podID="5306d4a8-24eb-461e-9384-f71ed3b41907" containerID="c985732a419721a10ec51507f3b1074efca8689dbaa00b0ec64495104289868b" exitCode=255 Oct 30 15:38:06.084084 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.084048 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" event={"ID":"5306d4a8-24eb-461e-9384-f71ed3b41907","Type":"ContainerDied","Data":"c985732a419721a10ec51507f3b1074efca8689dbaa00b0ec64495104289868b"} Oct 30 15:38:06.084084 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.084080 5013 scope.go:122] "RemoveContainer" containerID="f0bff60ba47b0b67c8bd6bf33c99bf64c08bc4973fb67c080a9b011c64cfbfd2" Oct 30 15:38:06.084426 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.084408 5013 scope.go:122] "RemoveContainer" containerID="c985732a419721a10ec51507f3b1074efca8689dbaa00b0ec64495104289868b" Oct 30 15:38:06.084637 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:06.084613 5013 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tj9gm_openshift-console-operator(5306d4a8-24eb-461e-9384-f71ed3b41907)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" podUID="5306d4a8-24eb-461e-9384-f71ed3b41907" Oct 30 15:38:06.085643 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.085624 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" event={"ID":"4345f2dd-427e-40d1-a5cc-43440c8cd4f0","Type":"ContainerStarted","Data":"f355e836b8794bba1361bf7aee4bcaee976be675d8015eb5a4d7c986d167967d"} Oct 30 15:38:06.085767 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.085750 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" event={"ID":"4345f2dd-427e-40d1-a5cc-43440c8cd4f0","Type":"ContainerStarted","Data":"cc074f279c00686cef738b2cd78f5f5d6782b28b4839cc4024d16a4112eff5f6"} Oct 30 15:38:06.085903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.085889 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:06.154291 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.154259 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:06.451824 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.451797 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:38:06.542027 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.541994 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:06.571466 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.571442 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7ff55fd746-8jqtl_4345f2dd-427e-40d1-a5cc-43440c8cd4f0/registry/0.log" Oct 30 15:38:06.750974 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.748712 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:38:06.750974 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.748770 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:06.750974 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.748807 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:06.750974 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.748832 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:38:06.750974 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:06.748929 5013 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 15:38:06.750974 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:06.748984 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert podName:7322d7b1-561a-402f-a9bb-111264b04743 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:08.748967592 +0000 UTC m=+4.899864617 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-tstct" (UID: "7322d7b1-561a-402f-a9bb-111264b04743") : secret "networking-console-plugin-cert" not found Oct 30 15:38:06.750974 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:06.749041 5013 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 15:38:06.750974 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:06.749071 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls podName:e54831c9-77f6-4bce-b8fc-bbfd1838dddd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:08.749061859 +0000 UTC m=+4.899958883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-xqb8n" (UID: "e54831c9-77f6-4bce-b8fc-bbfd1838dddd") : secret "cluster-monitoring-operator-tls" not found Oct 30 15:38:06.750974 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:06.749123 5013 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 15:38:06.750974 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:06.749152 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs podName:0a1bfd21-ddfa-4820-8356-49214c4cfc3b nodeName:}" failed. No retries permitted until 2026-10-30 15:38:08.7491412 +0000 UTC m=+4.900038215 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs") pod "router-default-779688d9fc-m8g4f" (UID: "0a1bfd21-ddfa-4820-8356-49214c4cfc3b") : secret "router-metrics-certs-default" not found Oct 30 15:38:06.750974 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:06.749218 5013 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 15:38:06.750974 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:06.749249 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls podName:b9bcd436-d777-4ca7-a840-bce674d247fd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:08.749239341 +0000 UTC m=+4.900136364 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g47gl" (UID: "b9bcd436-d777-4ca7-a840-bce674d247fd") : secret "insights-runtime-extractor-tls" not found Oct 30 15:38:06.773613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:06.773584 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-857f96f97f-7lt52_92898f00-aba8-475c-9389-1f4d3fae8103/registry/0.log" Oct 30 15:38:07.093100 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:07.093016 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c6b9v" event={"ID":"cf86f01f-c44d-46a3-be13-03c99b006285","Type":"ContainerStarted","Data":"6b72e590b012b0b6b0858b3afc82d518bbee9e0941a4e8531f09998438b6490e"} Oct 30 15:38:07.093100 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:07.093080 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c6b9v" event={"ID":"cf86f01f-c44d-46a3-be13-03c99b006285","Type":"ContainerStarted","Data":"a79972ea00d5a508c2372e1f9320c8be2c912c7a85266eabd25cc64821d079c5"} Oct 30 15:38:07.095096 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:07.095068 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 15:38:07.095504 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:07.095481 5013 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 15:38:07.172136 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:07.172111 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nvxqq_3173666e-bb4d-4f89-99bd-061c19370b91/node-ca/0.log" Oct 30 15:38:07.174294 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:07.174263 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:07.342884 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:07.342839 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:07.772358 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:07.772299 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hrsgx_910e7c90-9822-46c7-aa83-929c32dee95c/serve-healthcheck-canary/0.log" Oct 30 15:38:07.830123 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:07.830087 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:07.999405 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:07.999374 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:08.100825 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.100750 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bbfpq" event={"ID":"f77495d2-eb88-48cb-b46e-510fe055471b","Type":"ContainerStarted","Data":"969f03c04e7436b4689b87b6a37d01cb6caf74bec08b24e37102c6a1f08fe766"} Oct 30 15:38:08.100825 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.100805 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bbfpq" event={"ID":"f77495d2-eb88-48cb-b46e-510fe055471b","Type":"ContainerStarted","Data":"5f407f7fdc4f7aa4c984d1697141a9aabaca77d8a0de10d6d706846525069438"} Oct 30 15:38:08.101300 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.100879 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bbfpq" Oct 30 15:38:08.102714 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.102691 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" event={"ID":"9eeaa1d3-abc9-4811-acad-f3a4419d967e","Type":"ContainerStarted","Data":"be96c10abf23f717e7bfd83d779c30d160c82bd7c63b012f3439ae6f0bc78e1e"} Oct 30 15:38:08.102807 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.102727 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-w2kcd" event={"ID":"9eeaa1d3-abc9-4811-acad-f3a4419d967e","Type":"ContainerStarted","Data":"18ab3ce2c732193041b34e158ed65296ea871bebb9389397586e6a6441df9d0e"} Oct 30 15:38:08.170054 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.170029 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:08.171604 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.171589 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-fv7tg_521c6b52-045a-447e-9d9e-49c0de21c0a2/migrator/0.log" Oct 30 15:38:08.374341 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.374288 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-jzd98_50c89e19-a6a1-4abe-906d-e2548c00e23a/migrator/0.log" Oct 30 15:38:08.543026 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.542995 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:08.572407 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.572380 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr_eb4bb521-d604-4dbe-b464-46fb320253a5/kube-storage-version-migrator-operator/0.log" Oct 30 15:38:08.765820 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.765797 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:38:08.765947 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.765840 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:08.765947 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.765891 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:08.765947 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.765927 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:38:08.766087 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:08.765951 5013 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 15:38:08.766087 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:08.766014 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls podName:e54831c9-77f6-4bce-b8fc-bbfd1838dddd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:12.765998567 +0000 UTC m=+8.916895597 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-xqb8n" (UID: "e54831c9-77f6-4bce-b8fc-bbfd1838dddd") : secret "cluster-monitoring-operator-tls" not found Oct 30 15:38:08.766087 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:08.766043 5013 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 15:38:08.766087 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:08.766077 5013 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 15:38:08.766272 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:08.766102 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs podName:0a1bfd21-ddfa-4820-8356-49214c4cfc3b nodeName:}" failed. No retries permitted until 2026-10-30 15:38:12.766086404 +0000 UTC m=+8.916983414 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs") pod "router-default-779688d9fc-m8g4f" (UID: "0a1bfd21-ddfa-4820-8356-49214c4cfc3b") : secret "router-metrics-certs-default" not found Oct 30 15:38:08.766272 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:08.766121 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert podName:7322d7b1-561a-402f-a9bb-111264b04743 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:12.766112344 +0000 UTC m=+8.917009353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-tstct" (UID: "7322d7b1-561a-402f-a9bb-111264b04743") : secret "networking-console-plugin-cert" not found Oct 30 15:38:08.766272 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:08.766158 5013 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 15:38:08.766272 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:08.766189 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls podName:b9bcd436-d777-4ca7-a840-bce674d247fd nodeName:}" failed. No retries permitted until 2026-10-30 15:38:12.766177761 +0000 UTC m=+8.917074813 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g47gl" (UID: "b9bcd436-d777-4ca7-a840-bce674d247fd") : secret "insights-runtime-extractor-tls" not found Oct 30 15:38:08.998930 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:08.998907 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:09.351671 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:09.351648 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:09.500972 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:09.500946 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:09.747763 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:09.747735 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:09.949011 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:09.948989 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:10.110495 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:10.110423 5013 generic.go:356] "Generic (PLEG): container finished" podID="2d95d50f-c563-4080-909a-8ec8dc3b69b7" containerID="5d317a217cff3b1c3acf014be915a9f3d56ce1429225596913cf33f050eaa0f0" exitCode=0 Oct 30 15:38:10.110495 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:10.110464 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8bj4r" event={"ID":"2d95d50f-c563-4080-909a-8ec8dc3b69b7","Type":"ContainerDied","Data":"5d317a217cff3b1c3acf014be915a9f3d56ce1429225596913cf33f050eaa0f0"} Oct 30 15:38:10.110740 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:10.110725 5013 scope.go:122] "RemoveContainer" containerID="5d317a217cff3b1c3acf014be915a9f3d56ce1429225596913cf33f050eaa0f0" Oct 30 15:38:10.251411 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:10.251376 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:10.411705 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:10.411679 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:10.523825 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:10.523794 5013 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:38:10.524206 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:10.524182 5013 scope.go:122] "RemoveContainer" containerID="c985732a419721a10ec51507f3b1074efca8689dbaa00b0ec64495104289868b" Oct 30 15:38:10.524392 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:10.524372 5013 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tj9gm_openshift-console-operator(5306d4a8-24eb-461e-9384-f71ed3b41907)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" podUID="5306d4a8-24eb-461e-9384-f71ed3b41907" Oct 30 15:38:10.571740 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:10.571712 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:11.115943 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:11.115911 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8bj4r" event={"ID":"2d95d50f-c563-4080-909a-8ec8dc3b69b7","Type":"ContainerStarted","Data":"d13bbaae35499635562b85e6743a559409d1bfb28d1dc1690f4049dd574b9589"} Oct 30 15:38:11.345110 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:11.345074 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:11.760154 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:11.760127 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:11.919573 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:11.919543 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:12.123409 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:12.123332 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:12.285807 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:12.285776 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:12.450059 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:12.450034 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:12.801744 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:12.801667 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:12.801744 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:12.801709 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:12.801744 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:12.801736 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:38:12.802163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:12.801775 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:38:12.804442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:12.804397 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9bcd436-d777-4ca7-a840-bce674d247fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g47gl\" (UID: \"b9bcd436-d777-4ca7-a840-bce674d247fd\") " pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:12.804830 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:12.804813 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7322d7b1-561a-402f-a9bb-111264b04743-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-tstct\" (UID: \"7322d7b1-561a-402f-a9bb-111264b04743\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:38:12.805021 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:12.805001 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1bfd21-ddfa-4820-8356-49214c4cfc3b-metrics-certs\") pod \"router-default-779688d9fc-m8g4f\" (UID: \"0a1bfd21-ddfa-4820-8356-49214c4cfc3b\") " pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:12.805056 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:12.805001 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e54831c9-77f6-4bce-b8fc-bbfd1838dddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-xqb8n\" (UID: \"e54831c9-77f6-4bce-b8fc-bbfd1838dddd\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:38:12.805726 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:12.805709 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" Oct 30 15:38:12.815864 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:12.815848 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g47gl" Oct 30 15:38:13.102935 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:13.102869 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:13.103925 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:13.103909 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" Oct 30 15:38:13.121238 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:13.121179 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g47gl"] Oct 30 15:38:13.124630 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:13.124597 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9bcd436_d777_4ca7_a840_bce674d247fd.slice/crio-7792cdd837fc8053bbba8c3825ac161944e79943e57a9dc0b97657391c02a770 WatchSource:0}: Error finding container 7792cdd837fc8053bbba8c3825ac161944e79943e57a9dc0b97657391c02a770: Status 404 returned error can't find the container with id 7792cdd837fc8053bbba8c3825ac161944e79943e57a9dc0b97657391c02a770 Oct 30 15:38:13.133007 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:13.132977 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-tstct"] Oct 30 15:38:13.135740 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:13.135714 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7322d7b1_561a_402f_a9bb_111264b04743.slice/crio-5d3bb50ad0d680b7c53a6297c7e6fe10f2f2087a5836a7ad88f546dab79a6e6f WatchSource:0}: Error finding container 5d3bb50ad0d680b7c53a6297c7e6fe10f2f2087a5836a7ad88f546dab79a6e6f: Status 404 returned error can't find the container with id 5d3bb50ad0d680b7c53a6297c7e6fe10f2f2087a5836a7ad88f546dab79a6e6f Oct 30 15:38:13.409543 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:13.409522 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n"] Oct 30 15:38:13.411235 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:13.411208 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode54831c9_77f6_4bce_b8fc_bbfd1838dddd.slice/crio-893f0a26176afdf0583be156d043f7d831dc6da52bea9484dbb23fc170531a1e WatchSource:0}: Error finding container 893f0a26176afdf0583be156d043f7d831dc6da52bea9484dbb23fc170531a1e: Status 404 returned error can't find the container with id 893f0a26176afdf0583be156d043f7d831dc6da52bea9484dbb23fc170531a1e Oct 30 15:38:13.424759 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:13.424739 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-779688d9fc-m8g4f"] Oct 30 15:38:13.425653 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:13.425626 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a1bfd21_ddfa_4820_8356_49214c4cfc3b.slice/crio-5adad1bc1b1ae7328a76c62f337317aaef8426605b2423e576a5adffe575bdf9 WatchSource:0}: Error finding container 5adad1bc1b1ae7328a76c62f337317aaef8426605b2423e576a5adffe575bdf9: Status 404 returned error can't find the container with id 5adad1bc1b1ae7328a76c62f337317aaef8426605b2423e576a5adffe575bdf9 Oct 30 15:38:14.106026 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:14.105786 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:14.106401 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:14.106054 5013 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 15:38:14.133897 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:14.133863 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" event={"ID":"e54831c9-77f6-4bce-b8fc-bbfd1838dddd","Type":"ContainerStarted","Data":"893f0a26176afdf0583be156d043f7d831dc6da52bea9484dbb23fc170531a1e"} Oct 30 15:38:14.135800 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:14.135768 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-779688d9fc-m8g4f" event={"ID":"0a1bfd21-ddfa-4820-8356-49214c4cfc3b","Type":"ContainerStarted","Data":"53dcbded0e891c071b920cbd523c2acc728ed9424cde0cda966b618f4174b2b4"} Oct 30 15:38:14.135931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:14.135814 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-779688d9fc-m8g4f" event={"ID":"0a1bfd21-ddfa-4820-8356-49214c4cfc3b","Type":"ContainerStarted","Data":"5adad1bc1b1ae7328a76c62f337317aaef8426605b2423e576a5adffe575bdf9"} Oct 30 15:38:14.138318 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:14.138289 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g47gl" event={"ID":"b9bcd436-d777-4ca7-a840-bce674d247fd","Type":"ContainerStarted","Data":"46c1c4d5aa21b5b37471f3e38ca6002553217a8b4f52fb577b14a1a63af24e2f"} Oct 30 15:38:14.138402 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:14.138347 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g47gl" event={"ID":"b9bcd436-d777-4ca7-a840-bce674d247fd","Type":"ContainerStarted","Data":"d4301e8bdc5c19bba5bc05a354c2df94faf1163ae5041d3ec72b52addf9a754d"} Oct 30 15:38:14.138402 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:14.138375 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g47gl" event={"ID":"b9bcd436-d777-4ca7-a840-bce674d247fd","Type":"ContainerStarted","Data":"7792cdd837fc8053bbba8c3825ac161944e79943e57a9dc0b97657391c02a770"} Oct 30 15:38:14.139800 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:14.139773 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" event={"ID":"7322d7b1-561a-402f-a9bb-111264b04743","Type":"ContainerStarted","Data":"5d3bb50ad0d680b7c53a6297c7e6fe10f2f2087a5836a7ad88f546dab79a6e6f"} Oct 30 15:38:14.279006 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:14.278967 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:38:14.279451 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:14.279434 5013 scope.go:122] "RemoveContainer" containerID="c985732a419721a10ec51507f3b1074efca8689dbaa00b0ec64495104289868b" Oct 30 15:38:14.279656 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:14.279631 5013 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tj9gm_openshift-console-operator(5306d4a8-24eb-461e-9384-f71ed3b41907)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" podUID="5306d4a8-24eb-461e-9384-f71ed3b41907" Oct 30 15:38:15.103185 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:15.103152 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:15.106087 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:15.106050 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:15.145544 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:15.145517 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-tstct" event={"ID":"7322d7b1-561a-402f-a9bb-111264b04743","Type":"ContainerStarted","Data":"c3baaffb26af07e230446601a3056c9c4cf8fe37c7caca0636d162a4816a4522"} Oct 30 15:38:15.785751 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:15.785721 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:15.961499 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:15.961463 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:15.998864 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:15.998839 5013 scope.go:122] "RemoveContainer" containerID="c985732a419721a10ec51507f3b1074efca8689dbaa00b0ec64495104289868b" Oct 30 15:38:16.150943 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.150899 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-xqb8n" event={"ID":"e54831c9-77f6-4bce-b8fc-bbfd1838dddd","Type":"ContainerStarted","Data":"74228646b123ae8e34a020c5ab0863db014dc82176b08210f929b322f0a8624b"} Oct 30 15:38:16.153182 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.153149 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g47gl" event={"ID":"b9bcd436-d777-4ca7-a840-bce674d247fd","Type":"ContainerStarted","Data":"d76e0bdce6e18fae0cd8b0bde0796286112bf166a3a33171a2ba73b729a7321c"} Oct 30 15:38:16.154977 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.154958 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 15:38:16.155075 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.155054 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" event={"ID":"5306d4a8-24eb-461e-9384-f71ed3b41907","Type":"ContainerStarted","Data":"86c034c0bac154b4069e0f658c170b2275abf81c39222c84520a0e9756ec09d9"} Oct 30 15:38:16.155334 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.155318 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:38:16.156464 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.156437 5013 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-tj9gm container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.20:8443/readyz\": dial tcp 10.132.0.20:8443: connect: connection refused" start-of-body= Oct 30 15:38:16.156721 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.156483 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" podUID="5306d4a8-24eb-461e-9384-f71ed3b41907" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.20:8443/readyz\": dial tcp 10.132.0.20:8443: connect: connection refused" Oct 30 15:38:16.157116 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.157095 5013 generic.go:356] "Generic (PLEG): container finished" podID="2d95d50f-c563-4080-909a-8ec8dc3b69b7" containerID="d13bbaae35499635562b85e6743a559409d1bfb28d1dc1690f4049dd574b9589" exitCode=0 Oct 30 15:38:16.157186 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.157160 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8bj4r" event={"ID":"2d95d50f-c563-4080-909a-8ec8dc3b69b7","Type":"ContainerDied","Data":"d13bbaae35499635562b85e6743a559409d1bfb28d1dc1690f4049dd574b9589"} Oct 30 15:38:16.157236 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.157225 5013 scope.go:122] "RemoveContainer" containerID="5d317a217cff3b1c3acf014be915a9f3d56ce1429225596913cf33f050eaa0f0" Oct 30 15:38:16.157409 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.157373 5013 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 15:38:16.157600 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.157585 5013 scope.go:122] "RemoveContainer" containerID="d13bbaae35499635562b85e6743a559409d1bfb28d1dc1690f4049dd574b9589" Oct 30 15:38:16.157802 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:16.157783 5013 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"insights-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=insights-operator pod=insights-operator-7c4986d449-8bj4r_openshift-insights(2d95d50f-c563-4080-909a-8ec8dc3b69b7)\"" pod="openshift-insights/insights-operator-7c4986d449-8bj4r" podUID="2d95d50f-c563-4080-909a-8ec8dc3b69b7" Oct 30 15:38:16.162709 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.162691 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:16.408390 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.408356 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:16.571573 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.571551 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:16.658096 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.657994 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:16.661116 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.661039 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-779688d9fc-m8g4f" Oct 30 15:38:16.670498 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.670477 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2dlp" Oct 30 15:38:16.899974 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:16.899940 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:17.087339 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:17.087254 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:17.167743 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:17.167715 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-tj9gm" Oct 30 15:38:17.190950 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:17.190921 5013 patch_prober.go:29] interesting pod/image-registry-857f96f97f-7lt52 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 30 15:38:17.191115 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:17.190971 5013 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-857f96f97f-7lt52" podUID="92898f00-aba8-475c-9389-1f4d3fae8103" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 15:38:17.287054 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:17.287026 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:17.483139 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:17.483116 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:18.108041 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:18.108015 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bbfpq" Oct 30 15:38:18.772439 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:18.772405 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-857f96f97f-7lt52"] Oct 30 15:38:19.001864 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:19.001782 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:19.175501 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:19.175475 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:19.393280 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:19.393249 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:20.391757 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:20.391726 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bbfpq_f77495d2-eb88-48cb-b46e-510fe055471b/dns/0.log" Oct 30 15:38:20.591880 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:20.591854 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bbfpq_f77495d2-eb88-48cb-b46e-510fe055471b/kube-rbac-proxy/0.log" Oct 30 15:38:21.592954 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:21.592927 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6k2tn_14a2267f-e095-45cb-b0bc-99d7ab20611a/dns-node-resolver/0.log" Oct 30 15:38:22.392953 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:22.392926 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7ff55fd746-8jqtl_4345f2dd-427e-40d1-a5cc-43440c8cd4f0/registry/0.log" Oct 30 15:38:22.591911 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:22.591886 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-857f96f97f-7lt52_92898f00-aba8-475c-9389-1f4d3fae8103/registry/0.log" Oct 30 15:38:22.991414 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:22.991392 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nvxqq_3173666e-bb4d-4f89-99bd-061c19370b91/node-ca/0.log" Oct 30 15:38:23.416037 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:23.416010 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-779688d9fc-m8g4f_0a1bfd21-ddfa-4820-8356-49214c4cfc3b/router/0.log" Oct 30 15:38:23.799511 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:23.799449 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hrsgx_910e7c90-9822-46c7-aa83-929c32dee95c/serve-healthcheck-canary/0.log" Oct 30 15:38:26.975222 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:26.975169 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:38:26.996650 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:26.996631 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g8vnr" Oct 30 15:38:27.100237 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:27.100215 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:38:27.465584 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:27.465551 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:27.644251 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:27.644216 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:28.019841 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:28.019807 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:28.189573 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:28.189539 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:28.387237 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:28.387213 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:28.572758 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:28.572726 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:28.777765 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:28.777706 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:29.998631 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:29.998600 5013 scope.go:122] "RemoveContainer" containerID="d13bbaae35499635562b85e6743a559409d1bfb28d1dc1690f4049dd574b9589" Oct 30 15:38:30.217829 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:30.217795 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8bj4r" event={"ID":"2d95d50f-c563-4080-909a-8ec8dc3b69b7","Type":"ContainerStarted","Data":"8890e773dc5f8bce19b4e81f5516cde4789f8a8163f5750b5ef8307833459ad2"} Oct 30 15:38:31.116619 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:31.116582 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:31.280242 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:31.280211 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:31.450409 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:31.450379 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:32.755175 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:32.755142 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:32.975435 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:32.975399 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:33.034875 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.034812 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xmpms"] Oct 30 15:38:33.066666 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.066642 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.101134 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.101087 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 15:38:33.101325 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.101308 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 15:38:33.113029 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.113005 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 15:38:33.113029 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.113009 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 15:38:33.113183 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.113145 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bdgdg\"" Oct 30 15:38:33.160368 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.160331 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:33.172511 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.172488 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/99088f87-9a18-44c4-b844-8c14d7b11100-root\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.172629 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.172517 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-accelerators-collector-config\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.172629 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.172555 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.172748 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.172694 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99088f87-9a18-44c4-b844-8c14d7b11100-sys\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.172748 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.172736 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-wtmp\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.172828 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.172789 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-tls\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.172828 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.172820 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-textfile\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.172899 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.172844 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjhj6\" (UniqueName: \"kubernetes.io/projected/99088f87-9a18-44c4-b844-8c14d7b11100-kube-api-access-fjhj6\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.172899 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.172860 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99088f87-9a18-44c4-b844-8c14d7b11100-metrics-client-ca\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.273504 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.273478 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/99088f87-9a18-44c4-b844-8c14d7b11100-root\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.273504 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.273506 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-accelerators-collector-config\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.273693 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.273551 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.273693 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.273585 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/99088f87-9a18-44c4-b844-8c14d7b11100-root\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.273693 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.273593 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99088f87-9a18-44c4-b844-8c14d7b11100-sys\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.273693 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.273635 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99088f87-9a18-44c4-b844-8c14d7b11100-sys\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.273693 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.273635 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-wtmp\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.273693 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.273669 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-tls\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.273693 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.273692 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-textfile\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.274046 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.273721 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjhj6\" (UniqueName: \"kubernetes.io/projected/99088f87-9a18-44c4-b844-8c14d7b11100-kube-api-access-fjhj6\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.274046 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.273744 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-wtmp\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.274046 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.273747 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99088f87-9a18-44c4-b844-8c14d7b11100-metrics-client-ca\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.274222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.274097 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-textfile\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.274281 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.274221 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-accelerators-collector-config\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.274281 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.274245 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99088f87-9a18-44c4-b844-8c14d7b11100-metrics-client-ca\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.276117 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.276097 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.276207 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.276173 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/99088f87-9a18-44c4-b844-8c14d7b11100-node-exporter-tls\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.298102 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.298056 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjhj6\" (UniqueName: \"kubernetes.io/projected/99088f87-9a18-44c4-b844-8c14d7b11100-kube-api-access-fjhj6\") pod \"node-exporter-xmpms\" (UID: \"99088f87-9a18-44c4-b844-8c14d7b11100\") " pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.405607 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.405572 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:33.419283 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.419257 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xmpms" Oct 30 15:38:33.449389 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:38:33.449357 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99088f87_9a18_44c4_b844_8c14d7b11100.slice/crio-7da399df66b9d376a5f2315b8e37ca69ef7456adb0da6976b53dfc5705355a91 WatchSource:0}: Error finding container 7da399df66b9d376a5f2315b8e37ca69ef7456adb0da6976b53dfc5705355a91: Status 404 returned error can't find the container with id 7da399df66b9d376a5f2315b8e37ca69ef7456adb0da6976b53dfc5705355a91 Oct 30 15:38:33.567554 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:33.567501 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:34.179599 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:34.179565 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:34.234689 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:34.234650 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xmpms" event={"ID":"99088f87-9a18-44c4-b844-8c14d7b11100","Type":"ContainerStarted","Data":"7da399df66b9d376a5f2315b8e37ca69ef7456adb0da6976b53dfc5705355a91"} Oct 30 15:38:34.355719 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:34.355682 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:34.583725 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:34.583657 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:35.240566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:35.240531 5013 generic.go:356] "Generic (PLEG): container finished" podID="99088f87-9a18-44c4-b844-8c14d7b11100" containerID="2105c80d2e6f3fe0833ae1c44aa48d4ba411071818826759d9202b2f6bdd9bcb" exitCode=0 Oct 30 15:38:35.240941 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:35.240618 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xmpms" event={"ID":"99088f87-9a18-44c4-b844-8c14d7b11100","Type":"ContainerDied","Data":"2105c80d2e6f3fe0833ae1c44aa48d4ba411071818826759d9202b2f6bdd9bcb"} Oct 30 15:38:35.297819 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:35.297796 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:35.465433 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:35.465403 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:36.247669 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:36.247634 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xmpms" event={"ID":"99088f87-9a18-44c4-b844-8c14d7b11100","Type":"ContainerStarted","Data":"213308a06db0119f7ea0cf55d58abb044a78897416ca87702ec7a7a96763f16c"} Oct 30 15:38:36.248129 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:36.247689 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xmpms" event={"ID":"99088f87-9a18-44c4-b844-8c14d7b11100","Type":"ContainerStarted","Data":"635add097fd361429f5a72efa63d38e31a6ed932dfdd83add6527566b1c07588"} Oct 30 15:38:36.320149 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:36.320086 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xmpms" podStartSLOduration=1.9255565350000001 podStartE2EDuration="3.320073194s" totalImagesPullingTime="1.311272252s" totalInitContainerRuntime="83.244407ms" isStatefulPod=true podCreationTimestamp="2026-10-30 15:38:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:36.308089864 +0000 UTC m=+32.458986896" watchObservedRunningTime="2026-10-30 15:38:36.320073194 +0000 UTC m=+32.470970260" Oct 30 15:38:36.337253 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:36.337224 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:36.455488 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:36.455462 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5dlc4" Oct 30 15:38:36.548626 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:36.548562 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:36.748161 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:36.748131 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:37.725947 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:37.725909 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7ff55fd746-8jqtl"] Oct 30 15:38:37.847671 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:37.847646 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:39.045729 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:39.045702 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:39.207114 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:39.207085 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:40.063212 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:40.063171 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:42.048714 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:42.048686 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:42.217973 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:42.217938 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:43.793600 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:43.793510 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-857f96f97f-7lt52" podUID="92898f00-aba8-475c-9389-1f4d3fae8103" containerName="registry" containerID="cri-o://e9d722e2feb74e14470826b9c40e43b76032adb41715a42adbd22e31840ab5a2" gracePeriod=30 Oct 30 15:38:44.037563 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.037541 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:44.169004 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.168976 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-image-registry-private-configuration\") pod \"92898f00-aba8-475c-9389-1f4d3fae8103\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " Oct 30 15:38:44.169179 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.169015 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-installation-pull-secrets\") pod \"92898f00-aba8-475c-9389-1f4d3fae8103\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " Oct 30 15:38:44.169179 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.169039 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92898f00-aba8-475c-9389-1f4d3fae8103-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92898f00-aba8-475c-9389-1f4d3fae8103-ca-trust-extracted\") pod \"92898f00-aba8-475c-9389-1f4d3fae8103\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " Oct 30 15:38:44.169179 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.169067 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-bound-sa-token\") pod \"92898f00-aba8-475c-9389-1f4d3fae8103\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " Oct 30 15:38:44.169179 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.169091 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-kube-api-access-v8xdt\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-kube-api-access-v8xdt\") pod \"92898f00-aba8-475c-9389-1f4d3fae8103\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " Oct 30 15:38:44.169179 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.169111 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-registry-certificates\") pod \"92898f00-aba8-475c-9389-1f4d3fae8103\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " Oct 30 15:38:44.169179 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.169148 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls\") pod \"92898f00-aba8-475c-9389-1f4d3fae8103\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " Oct 30 15:38:44.169179 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.169178 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-trusted-ca\") pod \"92898f00-aba8-475c-9389-1f4d3fae8103\" (UID: \"92898f00-aba8-475c-9389-1f4d3fae8103\") " Oct 30 15:38:44.169644 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.169615 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-registry-certificates" pod "92898f00-aba8-475c-9389-1f4d3fae8103" (UID: "92898f00-aba8-475c-9389-1f4d3fae8103"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:38:44.169710 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.169686 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-trusted-ca" pod "92898f00-aba8-475c-9389-1f4d3fae8103" (UID: "92898f00-aba8-475c-9389-1f4d3fae8103"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:38:44.171676 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.171647 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-image-registry-private-configuration" pod "92898f00-aba8-475c-9389-1f4d3fae8103" (UID: "92898f00-aba8-475c-9389-1f4d3fae8103"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:38:44.172056 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.172017 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-kube-api-access-v8xdt" pod "92898f00-aba8-475c-9389-1f4d3fae8103" (UID: "92898f00-aba8-475c-9389-1f4d3fae8103"). InnerVolumeSpecName "kube-api-access-v8xdt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:38:44.172165 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.172096 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-bound-sa-token" pod "92898f00-aba8-475c-9389-1f4d3fae8103" (UID: "92898f00-aba8-475c-9389-1f4d3fae8103"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:38:44.172252 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.172185 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls" pod "92898f00-aba8-475c-9389-1f4d3fae8103" (UID: "92898f00-aba8-475c-9389-1f4d3fae8103"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:38:44.172313 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.172288 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-installation-pull-secrets" pod "92898f00-aba8-475c-9389-1f4d3fae8103" (UID: "92898f00-aba8-475c-9389-1f4d3fae8103"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:38:44.175675 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.175647 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92898f00-aba8-475c-9389-1f4d3fae8103-ca-trust-extracted" pod "92898f00-aba8-475c-9389-1f4d3fae8103" (UID: "92898f00-aba8-475c-9389-1f4d3fae8103"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:38:44.270400 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.270370 5013 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-bound-sa-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:38:44.270400 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.270396 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8xdt\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-kube-api-access-v8xdt\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:38:44.270551 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.270411 5013 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-registry-certificates\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:38:44.270551 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.270424 5013 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92898f00-aba8-475c-9389-1f4d3fae8103-registry-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:38:44.270551 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.270435 5013 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92898f00-aba8-475c-9389-1f4d3fae8103-trusted-ca\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:38:44.270551 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.270450 5013 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-image-registry-private-configuration\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:38:44.270551 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.270463 5013 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92898f00-aba8-475c-9389-1f4d3fae8103-installation-pull-secrets\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:38:44.270551 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.270475 5013 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92898f00-aba8-475c-9389-1f4d3fae8103-ca-trust-extracted\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:38:44.278707 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.278684 5013 generic.go:356] "Generic (PLEG): container finished" podID="92898f00-aba8-475c-9389-1f4d3fae8103" containerID="e9d722e2feb74e14470826b9c40e43b76032adb41715a42adbd22e31840ab5a2" exitCode=0 Oct 30 15:38:44.278802 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.278727 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-857f96f97f-7lt52" Oct 30 15:38:44.278802 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.278766 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-857f96f97f-7lt52" event={"ID":"92898f00-aba8-475c-9389-1f4d3fae8103","Type":"ContainerDied","Data":"e9d722e2feb74e14470826b9c40e43b76032adb41715a42adbd22e31840ab5a2"} Oct 30 15:38:44.278899 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.278810 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-857f96f97f-7lt52" event={"ID":"92898f00-aba8-475c-9389-1f4d3fae8103","Type":"ContainerDied","Data":"b43483199697c2ba06e9a2848696f842b28011c5d0a35c572cf0ea76ee3e6f7b"} Oct 30 15:38:44.278899 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.278834 5013 scope.go:122] "RemoveContainer" containerID="e9d722e2feb74e14470826b9c40e43b76032adb41715a42adbd22e31840ab5a2" Oct 30 15:38:44.287280 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.287265 5013 scope.go:122] "RemoveContainer" containerID="e9d722e2feb74e14470826b9c40e43b76032adb41715a42adbd22e31840ab5a2" Oct 30 15:38:44.287530 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:44.287510 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9d722e2feb74e14470826b9c40e43b76032adb41715a42adbd22e31840ab5a2\": container with ID starting with e9d722e2feb74e14470826b9c40e43b76032adb41715a42adbd22e31840ab5a2 not found: ID does not exist" containerID="e9d722e2feb74e14470826b9c40e43b76032adb41715a42adbd22e31840ab5a2" Oct 30 15:38:44.287581 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.287539 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9d722e2feb74e14470826b9c40e43b76032adb41715a42adbd22e31840ab5a2"} err="failed to get container status \"e9d722e2feb74e14470826b9c40e43b76032adb41715a42adbd22e31840ab5a2\": rpc error: code = NotFound desc = could not find container \"e9d722e2feb74e14470826b9c40e43b76032adb41715a42adbd22e31840ab5a2\": container with ID starting with e9d722e2feb74e14470826b9c40e43b76032adb41715a42adbd22e31840ab5a2 not found: ID does not exist" Oct 30 15:38:44.329375 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.329347 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-857f96f97f-7lt52"] Oct 30 15:38:44.333453 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:44.333436 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-857f96f97f-7lt52"] Oct 30 15:38:46.004288 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:46.004254 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92898f00-aba8-475c-9389-1f4d3fae8103" path="/var/lib/kubelet/pods/92898f00-aba8-475c-9389-1f4d3fae8103/volumes" Oct 30 15:38:46.078877 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:46.078847 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:46.285361 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:46.285269 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:46.493424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:46.493385 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:53.154615 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:53.154575 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:53.360819 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:53.360795 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:56.975788 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:38:56.975749 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:38:58.086931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:58.086888 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:38:58.251232 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:38:58.251189 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:00.488722 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:00.488658 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:00.648399 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:00.648369 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:01.377363 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:01.377329 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:01.536297 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:01.536265 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:02.752681 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:02.752642 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" podUID="4345f2dd-427e-40d1-a5cc-43440c8cd4f0" containerName="registry" containerID="cri-o://f355e836b8794bba1361bf7aee4bcaee976be675d8015eb5a4d7c986d167967d" gracePeriod=30 Oct 30 15:39:02.996101 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:02.996077 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:39:03.108544 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.108459 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-certificates\") pod \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " Oct 30 15:39:03.108544 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.108527 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-kube-api-access-tbkl9\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-kube-api-access-tbkl9\") pod \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " Oct 30 15:39:03.108774 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.108558 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-installation-pull-secrets\") pod \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " Oct 30 15:39:03.108774 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.108593 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls\") pod \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " Oct 30 15:39:03.108774 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.108621 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-image-registry-private-configuration\") pod \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " Oct 30 15:39:03.108774 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.108757 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-bound-sa-token\") pod \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " Oct 30 15:39:03.108986 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.108829 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-ca-trust-extracted\") pod \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " Oct 30 15:39:03.108986 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.108890 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-trusted-ca\") pod \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\" (UID: \"4345f2dd-427e-40d1-a5cc-43440c8cd4f0\") " Oct 30 15:39:03.108986 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.108916 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-certificates" pod "4345f2dd-427e-40d1-a5cc-43440c8cd4f0" (UID: "4345f2dd-427e-40d1-a5cc-43440c8cd4f0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:39:03.109144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.109124 5013 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-certificates\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:39:03.109678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.109647 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-trusted-ca" pod "4345f2dd-427e-40d1-a5cc-43440c8cd4f0" (UID: "4345f2dd-427e-40d1-a5cc-43440c8cd4f0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:39:03.111218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.111164 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-image-registry-private-configuration" pod "4345f2dd-427e-40d1-a5cc-43440c8cd4f0" (UID: "4345f2dd-427e-40d1-a5cc-43440c8cd4f0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:39:03.111319 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.111245 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls" pod "4345f2dd-427e-40d1-a5cc-43440c8cd4f0" (UID: "4345f2dd-427e-40d1-a5cc-43440c8cd4f0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:39:03.111419 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.111401 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-installation-pull-secrets" pod "4345f2dd-427e-40d1-a5cc-43440c8cd4f0" (UID: "4345f2dd-427e-40d1-a5cc-43440c8cd4f0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:39:03.111576 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.111547 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-kube-api-access-tbkl9" pod "4345f2dd-427e-40d1-a5cc-43440c8cd4f0" (UID: "4345f2dd-427e-40d1-a5cc-43440c8cd4f0"). InnerVolumeSpecName "kube-api-access-tbkl9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:39:03.111853 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.111823 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-bound-sa-token" pod "4345f2dd-427e-40d1-a5cc-43440c8cd4f0" (UID: "4345f2dd-427e-40d1-a5cc-43440c8cd4f0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:39:03.115504 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.115387 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-ca-trust-extracted" pod "4345f2dd-427e-40d1-a5cc-43440c8cd4f0" (UID: "4345f2dd-427e-40d1-a5cc-43440c8cd4f0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:39:03.209896 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.209873 5013 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-registry-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:39:03.209997 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.209898 5013 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-image-registry-private-configuration\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:39:03.209997 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.209912 5013 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-bound-sa-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:39:03.209997 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.209925 5013 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-ca-trust-extracted\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:39:03.209997 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.209938 5013 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-trusted-ca\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:39:03.209997 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.209950 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbkl9\" (UniqueName: \"kubernetes.io/projected/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-kube-api-access-tbkl9\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:39:03.209997 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.209962 5013 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4345f2dd-427e-40d1-a5cc-43440c8cd4f0-installation-pull-secrets\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:39:03.352528 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.352500 5013 generic.go:356] "Generic (PLEG): container finished" podID="4345f2dd-427e-40d1-a5cc-43440c8cd4f0" containerID="f355e836b8794bba1361bf7aee4bcaee976be675d8015eb5a4d7c986d167967d" exitCode=0 Oct 30 15:39:03.352687 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.352559 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" event={"ID":"4345f2dd-427e-40d1-a5cc-43440c8cd4f0","Type":"ContainerDied","Data":"f355e836b8794bba1361bf7aee4bcaee976be675d8015eb5a4d7c986d167967d"} Oct 30 15:39:03.352687 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.352565 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" Oct 30 15:39:03.352687 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.352608 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7ff55fd746-8jqtl" event={"ID":"4345f2dd-427e-40d1-a5cc-43440c8cd4f0","Type":"ContainerDied","Data":"cc074f279c00686cef738b2cd78f5f5d6782b28b4839cc4024d16a4112eff5f6"} Oct 30 15:39:03.352687 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.352628 5013 scope.go:122] "RemoveContainer" containerID="f355e836b8794bba1361bf7aee4bcaee976be675d8015eb5a4d7c986d167967d" Oct 30 15:39:03.360987 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.360970 5013 scope.go:122] "RemoveContainer" containerID="f355e836b8794bba1361bf7aee4bcaee976be675d8015eb5a4d7c986d167967d" Oct 30 15:39:03.361288 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:39:03.361270 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f355e836b8794bba1361bf7aee4bcaee976be675d8015eb5a4d7c986d167967d\": container with ID starting with f355e836b8794bba1361bf7aee4bcaee976be675d8015eb5a4d7c986d167967d not found: ID does not exist" containerID="f355e836b8794bba1361bf7aee4bcaee976be675d8015eb5a4d7c986d167967d" Oct 30 15:39:03.361345 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.361296 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f355e836b8794bba1361bf7aee4bcaee976be675d8015eb5a4d7c986d167967d"} err="failed to get container status \"f355e836b8794bba1361bf7aee4bcaee976be675d8015eb5a4d7c986d167967d\": rpc error: code = NotFound desc = could not find container \"f355e836b8794bba1361bf7aee4bcaee976be675d8015eb5a4d7c986d167967d\": container with ID starting with f355e836b8794bba1361bf7aee4bcaee976be675d8015eb5a4d7c986d167967d not found: ID does not exist" Oct 30 15:39:03.402803 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.402772 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7ff55fd746-8jqtl"] Oct 30 15:39:03.414568 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:03.414543 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7ff55fd746-8jqtl"] Oct 30 15:39:04.002686 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:04.002648 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4345f2dd-427e-40d1-a5cc-43440c8cd4f0" path="/var/lib/kubelet/pods/4345f2dd-427e-40d1-a5cc-43440c8cd4f0/volumes" Oct 30 15:39:09.376780 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:09.376696 5013 generic.go:356] "Generic (PLEG): container finished" podID="eb4bb521-d604-4dbe-b464-46fb320253a5" containerID="872551740a91bb5a5bc3769875a9fda151fdc88d9421d63d36fcc19f6348b772" exitCode=0 Oct 30 15:39:09.376780 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:09.376759 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" event={"ID":"eb4bb521-d604-4dbe-b464-46fb320253a5","Type":"ContainerDied","Data":"872551740a91bb5a5bc3769875a9fda151fdc88d9421d63d36fcc19f6348b772"} Oct 30 15:39:09.377366 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:09.377127 5013 scope.go:122] "RemoveContainer" containerID="872551740a91bb5a5bc3769875a9fda151fdc88d9421d63d36fcc19f6348b772" Oct 30 15:39:10.382895 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:10.382855 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mmxcr" event={"ID":"eb4bb521-d604-4dbe-b464-46fb320253a5","Type":"ContainerStarted","Data":"9271ed063e95889058e27dae12fd19b7aed586dcdaceed4995b1774516b60d68"} Oct 30 15:39:14.328922 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:14.328891 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:16.425249 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:16.425217 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:16.595720 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:16.595689 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:18.968053 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:18.968012 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:23.492012 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:23.491982 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:23.652167 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:23.652139 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:23.843671 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:23.843600 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:26.171828 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:26.171797 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:26.334970 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:26.334940 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:26.975043 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:39:26.975007 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:39:29.455025 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:29.454993 5013 generic.go:356] "Generic (PLEG): container finished" podID="23eff76f-8eed-4d55-88b9-1ae63cb1e9ec" containerID="26783cef90471bbc7b1711c864ef301d264d485c12d4485b627b82496daaf929" exitCode=0 Oct 30 15:39:29.455447 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:29.455054 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" event={"ID":"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec","Type":"ContainerDied","Data":"26783cef90471bbc7b1711c864ef301d264d485c12d4485b627b82496daaf929"} Oct 30 15:39:29.455447 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:29.455374 5013 scope.go:122] "RemoveContainer" containerID="26783cef90471bbc7b1711c864ef301d264d485c12d4485b627b82496daaf929" Oct 30 15:39:30.461155 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:30.461119 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-ksnkn" event={"ID":"23eff76f-8eed-4d55-88b9-1ae63cb1e9ec","Type":"ContainerStarted","Data":"2ed0d865412e46b1b61a74c65d19e169d9b37ee9d1ef7ee5f8c67ffdbb9abb0b"} Oct 30 15:39:31.383567 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:31.383516 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:31.554619 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:31.554585 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:53.281058 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:39:53.281026 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:39:56.975682 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:39:56.975647 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:40:09.746249 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:40:09.746220 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:40:26.975923 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:40:26.975842 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:40:36.540476 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:40:36.540442 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:40:42.310473 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:40:42.310443 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:40:42.471689 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:40:42.471662 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:40:53.408263 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:40:53.408235 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:40:56.975861 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:40:56.975823 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:41:26.974659 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:41:26.974621 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:41:27.828435 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:41:27.828401 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:41:53.536407 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:41:53.536335 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:41:56.975237 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:41:56.975177 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:41:56.976767 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:41:56.976745 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:41:57.144718 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:41:57.144662 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:08.193611 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:08.193580 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:08.367316 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:08.367289 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:18.872547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:18.872512 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:19.747039 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:19.747007 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:22.072984 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:22.072947 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:22.239486 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:22.239350 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:24.752371 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:24.752337 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:24.948967 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:24.948939 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:26.820667 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:26.820636 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:26.975136 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:42:26.975102 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:42:27.008595 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:27.008563 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:27.343852 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:27.343822 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:29.069759 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:29.069725 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:29.235418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:29.235389 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:35.381161 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:35.381133 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:37.612914 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:37.612886 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:37.780210 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:37.780165 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:40.898425 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:40.898394 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:41.059757 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:41.059726 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:43.035650 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.035620 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-n8vzt"] Oct 30 15:42:43.036027 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.035904 5013 cpu_manager.go:414] "containerMap: removing container" podUID="4345f2dd-427e-40d1-a5cc-43440c8cd4f0" containerName="registry" Oct 30 15:42:43.036027 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.035914 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4345f2dd-427e-40d1-a5cc-43440c8cd4f0" containerName="registry" Oct 30 15:42:43.036027 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.035925 5013 cpu_manager.go:414] "containerMap: removing container" podUID="92898f00-aba8-475c-9389-1f4d3fae8103" containerName="registry" Oct 30 15:42:43.036027 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.035931 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92898f00-aba8-475c-9389-1f4d3fae8103" containerName="registry" Oct 30 15:42:43.036027 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.035982 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="4345f2dd-427e-40d1-a5cc-43440c8cd4f0" containerName="registry" Oct 30 15:42:43.036027 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.035988 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="92898f00-aba8-475c-9389-1f4d3fae8103" containerName="registry" Oct 30 15:42:43.038940 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.038925 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-8465994787-n8vzt" Oct 30 15:42:43.054152 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.054136 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-f6trr\"" Oct 30 15:42:43.066087 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.066060 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Oct 30 15:42:43.066240 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.066088 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Oct 30 15:42:43.068295 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.068257 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-n8vzt"] Oct 30 15:42:43.136514 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.136490 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d875h\" (UniqueName: \"kubernetes.io/projected/e7c63a13-3998-4993-b480-bb43d5d95826-kube-api-access-d875h\") pod \"openshift-lws-operator-8465994787-n8vzt\" (UID: \"e7c63a13-3998-4993-b480-bb43d5d95826\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-n8vzt" Oct 30 15:42:43.136616 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.136518 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7c63a13-3998-4993-b480-bb43d5d95826-tmp\") pod \"openshift-lws-operator-8465994787-n8vzt\" (UID: \"e7c63a13-3998-4993-b480-bb43d5d95826\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-n8vzt" Oct 30 15:42:43.236893 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.236863 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d875h\" (UniqueName: \"kubernetes.io/projected/e7c63a13-3998-4993-b480-bb43d5d95826-kube-api-access-d875h\") pod \"openshift-lws-operator-8465994787-n8vzt\" (UID: \"e7c63a13-3998-4993-b480-bb43d5d95826\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-n8vzt" Oct 30 15:42:43.237029 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.236896 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7c63a13-3998-4993-b480-bb43d5d95826-tmp\") pod \"openshift-lws-operator-8465994787-n8vzt\" (UID: \"e7c63a13-3998-4993-b480-bb43d5d95826\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-n8vzt" Oct 30 15:42:43.237323 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.237305 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7c63a13-3998-4993-b480-bb43d5d95826-tmp\") pod \"openshift-lws-operator-8465994787-n8vzt\" (UID: \"e7c63a13-3998-4993-b480-bb43d5d95826\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-n8vzt" Oct 30 15:42:43.260326 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.260306 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d875h\" (UniqueName: \"kubernetes.io/projected/e7c63a13-3998-4993-b480-bb43d5d95826-kube-api-access-d875h\") pod \"openshift-lws-operator-8465994787-n8vzt\" (UID: \"e7c63a13-3998-4993-b480-bb43d5d95826\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-n8vzt" Oct 30 15:42:43.362476 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.362408 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-8465994787-n8vzt" Oct 30 15:42:43.657758 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:43.657730 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-n8vzt"] Oct 30 15:42:43.659109 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:42:43.659083 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7c63a13_3998_4993_b480_bb43d5d95826.slice/crio-65b247f8313e2fef12a25256b39a9652ce272dcd621dcb8155aeef48cbf994cd WatchSource:0}: Error finding container 65b247f8313e2fef12a25256b39a9652ce272dcd621dcb8155aeef48cbf994cd: Status 404 returned error can't find the container with id 65b247f8313e2fef12a25256b39a9652ce272dcd621dcb8155aeef48cbf994cd Oct 30 15:42:44.186542 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:44.186509 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-8465994787-n8vzt" event={"ID":"e7c63a13-3998-4993-b480-bb43d5d95826","Type":"ContainerStarted","Data":"65b247f8313e2fef12a25256b39a9652ce272dcd621dcb8155aeef48cbf994cd"} Oct 30 15:42:51.220950 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:51.220908 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-8465994787-n8vzt" event={"ID":"e7c63a13-3998-4993-b480-bb43d5d95826","Type":"ContainerStarted","Data":"d2ce34c3c618c1f717a17d5ad7f7ae475a405959b801a30f52af3dc93b8b04f5"} Oct 30 15:42:51.267400 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:51.267355 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-8465994787-n8vzt" podStartSLOduration=1.05569245 podStartE2EDuration="8.267338826s" totalImagesPullingTime="7.211646376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:42:51.267066058 +0000 UTC m=+287.417963094" watchObservedRunningTime="2026-10-30 15:42:51.267338826 +0000 UTC m=+287.418235858" Oct 30 15:42:52.398605 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:52.398566 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:52.568509 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:52.568467 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:53.727575 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:53.727543 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:54.067676 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:42:54.067567 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:42:56.975250 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:42:56.975215 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:43:04.013125 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:04.013100 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 15:43:04.016783 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:04.016761 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 15:43:04.889153 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:04.889113 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:05.093997 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.093966 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:05.506749 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.506709 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:05.618017 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.617988 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p"] Oct 30 15:43:05.621683 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.621664 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.659008 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.658985 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p"] Oct 30 15:43:05.700263 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.700230 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7db72d9d-feff-40d4-997f-560e5f1b1ef1-manifests\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.700378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.700263 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xgkt\" (UniqueName: \"kubernetes.io/projected/7db72d9d-feff-40d4-997f-560e5f1b1ef1-kube-api-access-9xgkt\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.700419 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.700394 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7db72d9d-feff-40d4-997f-560e5f1b1ef1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.700453 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.700428 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7db72d9d-feff-40d4-997f-560e5f1b1ef1-tmp\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.700453 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.700450 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7db72d9d-feff-40d4-997f-560e5f1b1ef1-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.700588 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.700572 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7db72d9d-feff-40d4-997f-560e5f1b1ef1-metrics-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.726056 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.726036 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Oct 30 15:43:05.726176 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.726117 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Oct 30 15:43:05.726176 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.726129 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Oct 30 15:43:05.726340 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.726321 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gqp9p\"" Oct 30 15:43:05.726709 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.726690 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Oct 30 15:43:05.726814 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.726753 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\"" Oct 30 15:43:05.800921 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.800864 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7db72d9d-feff-40d4-997f-560e5f1b1ef1-manifests\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.800921 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.800889 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xgkt\" (UniqueName: \"kubernetes.io/projected/7db72d9d-feff-40d4-997f-560e5f1b1ef1-kube-api-access-9xgkt\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.801073 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.800924 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7db72d9d-feff-40d4-997f-560e5f1b1ef1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.801073 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.800944 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7db72d9d-feff-40d4-997f-560e5f1b1ef1-tmp\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.801073 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.800967 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7db72d9d-feff-40d4-997f-560e5f1b1ef1-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.801372 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.801116 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7db72d9d-feff-40d4-997f-560e5f1b1ef1-metrics-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.801670 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.801622 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7db72d9d-feff-40d4-997f-560e5f1b1ef1-manifests\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.808162 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.804292 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7db72d9d-feff-40d4-997f-560e5f1b1ef1-tmp\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.808162 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.804476 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7db72d9d-feff-40d4-997f-560e5f1b1ef1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.808162 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.806075 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7db72d9d-feff-40d4-997f-560e5f1b1ef1-metrics-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.808835 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.808814 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7db72d9d-feff-40d4-997f-560e5f1b1ef1-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.831928 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.831899 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xgkt\" (UniqueName: \"kubernetes.io/projected/7db72d9d-feff-40d4-997f-560e5f1b1ef1-kube-api-access-9xgkt\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p\" (UID: \"7db72d9d-feff-40d4-997f-560e5f1b1ef1\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:05.906448 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.906420 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:05.935561 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:05.935540 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:06.159825 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:06.159789 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:06.293864 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:06.291627 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p"] Oct 30 15:43:06.295018 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:43:06.294989 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db72d9d_feff_40d4_997f_560e5f1b1ef1.slice/crio-42fa0ed8d6d91b4368501dc30a06c9628c31102149fa5a901e5453bfcce84dad WatchSource:0}: Error finding container 42fa0ed8d6d91b4368501dc30a06c9628c31102149fa5a901e5453bfcce84dad: Status 404 returned error can't find the container with id 42fa0ed8d6d91b4368501dc30a06c9628c31102149fa5a901e5453bfcce84dad Oct 30 15:43:06.296791 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:06.296775 5013 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 15:43:06.347323 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:06.347298 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:07.283949 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:07.283917 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" event={"ID":"7db72d9d-feff-40d4-997f-560e5f1b1ef1","Type":"ContainerStarted","Data":"42fa0ed8d6d91b4368501dc30a06c9628c31102149fa5a901e5453bfcce84dad"} Oct 30 15:43:09.293179 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:09.293154 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" event={"ID":"7db72d9d-feff-40d4-997f-560e5f1b1ef1","Type":"ContainerStarted","Data":"257f7fc0a5a0e6ee2fbf066fa0624bfe14830a8089094efb06d99abec3cfbbf6"} Oct 30 15:43:10.298763 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:10.298723 5013 generic.go:356] "Generic (PLEG): container finished" podID="7db72d9d-feff-40d4-997f-560e5f1b1ef1" containerID="257f7fc0a5a0e6ee2fbf066fa0624bfe14830a8089094efb06d99abec3cfbbf6" exitCode=0 Oct 30 15:43:10.299133 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:10.298778 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" event={"ID":"7db72d9d-feff-40d4-997f-560e5f1b1ef1","Type":"ContainerDied","Data":"257f7fc0a5a0e6ee2fbf066fa0624bfe14830a8089094efb06d99abec3cfbbf6"} Oct 30 15:43:11.306623 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:11.306593 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" event={"ID":"7db72d9d-feff-40d4-997f-560e5f1b1ef1","Type":"ContainerStarted","Data":"e7240905d95c729c215765da3f9ba8d70cd019f2b4cf5ac914057c53d01d5538"} Oct 30 15:43:11.307050 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:11.306730 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:11.397975 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:11.397927 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" podStartSLOduration=2.590532574 podStartE2EDuration="6.397914204s" totalImagesPullingTime="3.313453283s" totalInitContainerRuntime="493.928347ms" isStatefulPod=false podCreationTimestamp="2026-10-30 15:43:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:43:11.385373718 +0000 UTC m=+307.536270748" watchObservedRunningTime="2026-10-30 15:43:11.397914204 +0000 UTC m=+307.548811232" Oct 30 15:43:22.314151 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:22.314070 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p" Oct 30 15:43:22.760481 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:22.760446 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:22.996627 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:22.996596 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:23.026859 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.026785 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx"] Oct 30 15:43:23.034867 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.034847 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:23.077247 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.077219 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx"] Oct 30 15:43:23.133894 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.133865 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Oct 30 15:43:23.134980 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.134965 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Oct 30 15:43:23.135483 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.135469 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Oct 30 15:43:23.135523 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.135475 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-w6qjz\"" Oct 30 15:43:23.139849 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.139829 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9-metrics-cert\") pod \"lws-controller-manager-7c5c6bfccb-k8lxx\" (UID: \"2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:23.139957 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.139864 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9-manager-config\") pod \"lws-controller-manager-7c5c6bfccb-k8lxx\" (UID: \"2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:23.139957 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.139883 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9-cert\") pod \"lws-controller-manager-7c5c6bfccb-k8lxx\" (UID: \"2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:23.140049 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.140030 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmvxp\" (UniqueName: \"kubernetes.io/projected/2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9-kube-api-access-rmvxp\") pod \"lws-controller-manager-7c5c6bfccb-k8lxx\" (UID: \"2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:23.240670 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.240641 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmvxp\" (UniqueName: \"kubernetes.io/projected/2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9-kube-api-access-rmvxp\") pod \"lws-controller-manager-7c5c6bfccb-k8lxx\" (UID: \"2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:23.240806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.240705 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9-metrics-cert\") pod \"lws-controller-manager-7c5c6bfccb-k8lxx\" (UID: \"2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:23.240806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.240731 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9-manager-config\") pod \"lws-controller-manager-7c5c6bfccb-k8lxx\" (UID: \"2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:23.240806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.240750 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9-cert\") pod \"lws-controller-manager-7c5c6bfccb-k8lxx\" (UID: \"2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:23.241438 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.241410 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9-manager-config\") pod \"lws-controller-manager-7c5c6bfccb-k8lxx\" (UID: \"2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:23.243436 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.243412 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9-metrics-cert\") pod \"lws-controller-manager-7c5c6bfccb-k8lxx\" (UID: \"2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:23.243605 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.243586 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9-cert\") pod \"lws-controller-manager-7c5c6bfccb-k8lxx\" (UID: \"2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:23.273879 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.273852 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmvxp\" (UniqueName: \"kubernetes.io/projected/2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9-kube-api-access-rmvxp\") pod \"lws-controller-manager-7c5c6bfccb-k8lxx\" (UID: \"2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:23.347039 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.346595 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:23.650095 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:23.650068 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx"] Oct 30 15:43:23.651374 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:43:23.651345 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ac166ee_a41e_4cb6_a3da_6b7b1914a9b9.slice/crio-1b8adb90affd040e68559c0db828e3d256bd9e99c132292fdaf46e70789d2d7a WatchSource:0}: Error finding container 1b8adb90affd040e68559c0db828e3d256bd9e99c132292fdaf46e70789d2d7a: Status 404 returned error can't find the container with id 1b8adb90affd040e68559c0db828e3d256bd9e99c132292fdaf46e70789d2d7a Oct 30 15:43:24.362725 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:24.362687 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" event={"ID":"2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9","Type":"ContainerStarted","Data":"1b8adb90affd040e68559c0db828e3d256bd9e99c132292fdaf46e70789d2d7a"} Oct 30 15:43:25.979864 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:25.979827 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:26.184349 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:26.184310 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:26.374270 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:26.374179 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" event={"ID":"2ac166ee-a41e-4cb6-a3da-6b7b1914a9b9","Type":"ContainerStarted","Data":"2eb1e9f3327d8e713dc071bfd55c13df480f6182316ceac468df99ae027e7c6b"} Oct 30 15:43:26.374270 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:26.374251 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:26.434032 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:26.433975 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" podStartSLOduration=2.102009011 podStartE2EDuration="4.433956459s" totalImagesPullingTime="2.331947448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:43:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:43:26.431955106 +0000 UTC m=+322.582852139" watchObservedRunningTime="2026-10-30 15:43:26.433956459 +0000 UTC m=+322.584853491" Oct 30 15:43:26.504441 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:26.504415 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:26.716335 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:26.716304 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:26.975789 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:43:26.975712 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:43:28.513335 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:28.513300 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:28.716030 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:28.716002 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:30.006415 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.006378 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:30.196669 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.196635 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-observability-584f99c788-9zvzt"] Oct 30 15:43:30.198928 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.198912 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-observability-584f99c788-9zvzt" Oct 30 15:43:30.223717 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.223697 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-observability-584f99c788-9zvzt"] Oct 30 15:43:30.230862 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.230836 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-observability-webhook-cert\"" Oct 30 15:43:30.230973 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.230870 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-observability-dockercfg-x9gnh\"" Oct 30 15:43:30.299507 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.299444 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/a9dfd31a-7eb0-4293-a813-be0cdcdea0f8-webhook-certs\") pod \"odh-observability-584f99c788-9zvzt\" (UID: \"a9dfd31a-7eb0-4293-a813-be0cdcdea0f8\") " pod="opendatahub/odh-observability-584f99c788-9zvzt" Oct 30 15:43:30.299507 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.299481 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvpwr\" (UniqueName: \"kubernetes.io/projected/a9dfd31a-7eb0-4293-a813-be0cdcdea0f8-kube-api-access-dvpwr\") pod \"odh-observability-584f99c788-9zvzt\" (UID: \"a9dfd31a-7eb0-4293-a813-be0cdcdea0f8\") " pod="opendatahub/odh-observability-584f99c788-9zvzt" Oct 30 15:43:30.316405 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.316379 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:30.400447 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.400417 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvpwr\" (UniqueName: \"kubernetes.io/projected/a9dfd31a-7eb0-4293-a813-be0cdcdea0f8-kube-api-access-dvpwr\") pod \"odh-observability-584f99c788-9zvzt\" (UID: \"a9dfd31a-7eb0-4293-a813-be0cdcdea0f8\") " pod="opendatahub/odh-observability-584f99c788-9zvzt" Oct 30 15:43:30.400613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.400477 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/a9dfd31a-7eb0-4293-a813-be0cdcdea0f8-webhook-certs\") pod \"odh-observability-584f99c788-9zvzt\" (UID: \"a9dfd31a-7eb0-4293-a813-be0cdcdea0f8\") " pod="opendatahub/odh-observability-584f99c788-9zvzt" Oct 30 15:43:30.400613 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:43:30.400570 5013 secret.go:189] Couldn't get secret opendatahub/odh-observability-webhook-cert: secret "odh-observability-webhook-cert" not found Oct 30 15:43:30.400706 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:43:30.400621 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dfd31a-7eb0-4293-a813-be0cdcdea0f8-webhook-certs podName:a9dfd31a-7eb0-4293-a813-be0cdcdea0f8 nodeName:}" failed. No retries permitted until 2026-10-30 15:43:30.900606885 +0000 UTC m=+327.051503895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-certs" (UniqueName: "kubernetes.io/secret/a9dfd31a-7eb0-4293-a813-be0cdcdea0f8-webhook-certs") pod "odh-observability-584f99c788-9zvzt" (UID: "a9dfd31a-7eb0-4293-a813-be0cdcdea0f8") : secret "odh-observability-webhook-cert" not found Oct 30 15:43:30.427019 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.426994 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvpwr\" (UniqueName: \"kubernetes.io/projected/a9dfd31a-7eb0-4293-a813-be0cdcdea0f8-kube-api-access-dvpwr\") pod \"odh-observability-584f99c788-9zvzt\" (UID: \"a9dfd31a-7eb0-4293-a813-be0cdcdea0f8\") " pod="opendatahub/odh-observability-584f99c788-9zvzt" Oct 30 15:43:30.492419 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.492388 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:30.674657 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.674630 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:30.903391 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.903363 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/a9dfd31a-7eb0-4293-a813-be0cdcdea0f8-webhook-certs\") pod \"odh-observability-584f99c788-9zvzt\" (UID: \"a9dfd31a-7eb0-4293-a813-be0cdcdea0f8\") " pod="opendatahub/odh-observability-584f99c788-9zvzt" Oct 30 15:43:30.906019 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:30.906002 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/a9dfd31a-7eb0-4293-a813-be0cdcdea0f8-webhook-certs\") pod \"odh-observability-584f99c788-9zvzt\" (UID: \"a9dfd31a-7eb0-4293-a813-be0cdcdea0f8\") " pod="opendatahub/odh-observability-584f99c788-9zvzt" Oct 30 15:43:31.109971 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:31.109903 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-observability-584f99c788-9zvzt" Oct 30 15:43:31.406522 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:31.406503 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-observability-584f99c788-9zvzt"] Oct 30 15:43:31.408107 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:43:31.408085 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9dfd31a_7eb0_4293_a813_be0cdcdea0f8.slice/crio-90d1c57f855af6e0482f16582d925c9dd91213945633606e7306d17482b6cd85 WatchSource:0}: Error finding container 90d1c57f855af6e0482f16582d925c9dd91213945633606e7306d17482b6cd85: Status 404 returned error can't find the container with id 90d1c57f855af6e0482f16582d925c9dd91213945633606e7306d17482b6cd85 Oct 30 15:43:32.330587 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:32.330547 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:32.398531 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:32.398500 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-observability-584f99c788-9zvzt" event={"ID":"a9dfd31a-7eb0-4293-a813-be0cdcdea0f8","Type":"ContainerStarted","Data":"90d1c57f855af6e0482f16582d925c9dd91213945633606e7306d17482b6cd85"} Oct 30 15:43:32.522922 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:32.522864 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:33.632999 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:33.632966 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:33.843941 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:33.843908 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:34.028717 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:34.028683 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:34.409429 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:34.409399 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-observability-584f99c788-9zvzt" event={"ID":"a9dfd31a-7eb0-4293-a813-be0cdcdea0f8","Type":"ContainerStarted","Data":"7877d08cd90626a7c56bde7e7eedd11eef97af4669da2f287b06a1a4c8eccc36"} Oct 30 15:43:34.409601 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:34.409546 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-observability-584f99c788-9zvzt" Oct 30 15:43:34.455811 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:34.455769 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-observability-584f99c788-9zvzt" podStartSLOduration=1.977879641 podStartE2EDuration="4.455755126s" totalImagesPullingTime="2.477875485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:43:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:43:34.454850482 +0000 UTC m=+330.605747510" watchObservedRunningTime="2026-10-30 15:43:34.455755126 +0000 UTC m=+330.606652157" Oct 30 15:43:37.380594 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:37.380551 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-k8lxx" Oct 30 15:43:43.567641 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.567610 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh"] Oct 30 15:43:43.571131 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.571115 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:43.597416 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.597394 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-hl6ks\"" Oct 30 15:43:43.600375 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.600355 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:43.604488 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.604469 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-q55hh\" (UID: \"072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:43.604562 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.604523 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khfrd\" (UniqueName: \"kubernetes.io/projected/072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83-kube-api-access-khfrd\") pod \"kserve-module-controller-manager-5cfb77bf8c-q55hh\" (UID: \"072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:43.604617 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.604594 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-q55hh\" (UID: \"072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:43.604654 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.604638 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-q55hh\" (UID: \"072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:43.621563 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.621539 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\"" Oct 30 15:43:43.622728 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.622707 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh"] Oct 30 15:43:43.705962 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.705926 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-q55hh\" (UID: \"072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:43.705962 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.705971 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-q55hh\" (UID: \"072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:43.706092 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.706015 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khfrd\" (UniqueName: \"kubernetes.io/projected/072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83-kube-api-access-khfrd\") pod \"kserve-module-controller-manager-5cfb77bf8c-q55hh\" (UID: \"072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:43.706092 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.706032 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-q55hh\" (UID: \"072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:43.706382 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.706366 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-q55hh\" (UID: \"072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:43.706455 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.706436 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-q55hh\" (UID: \"072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:43.707144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.707126 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-q55hh\" (UID: \"072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:43.729014 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.728993 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khfrd\" (UniqueName: \"kubernetes.io/projected/072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83-kube-api-access-khfrd\") pod \"kserve-module-controller-manager-5cfb77bf8c-q55hh\" (UID: \"072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:43.812144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.812120 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:43.883093 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:43.883065 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:44.211359 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:44.211335 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh"] Oct 30 15:43:44.212774 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:43:44.212743 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod072400c9_bf8c_42f7_a3f1_2b8bc4b3ac83.slice/crio-a4a7cdbf98eef86b461a9cfd75a3bc895d8d8a13359803f970c5c0edddc3e5b4 WatchSource:0}: Error finding container a4a7cdbf98eef86b461a9cfd75a3bc895d8d8a13359803f970c5c0edddc3e5b4: Status 404 returned error can't find the container with id a4a7cdbf98eef86b461a9cfd75a3bc895d8d8a13359803f970c5c0edddc3e5b4 Oct 30 15:43:44.448397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:44.448360 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" event={"ID":"072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83","Type":"ContainerStarted","Data":"a4a7cdbf98eef86b461a9cfd75a3bc895d8d8a13359803f970c5c0edddc3e5b4"} Oct 30 15:43:45.415757 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:45.415731 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-observability-584f99c788-9zvzt" Oct 30 15:43:47.462572 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:47.462532 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" event={"ID":"072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83","Type":"ContainerStarted","Data":"44459c8aa5f78dd2e27ec8e6a31b311b8a335fb6512c187afd4cc8ea37267faa"} Oct 30 15:43:47.463038 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:47.462606 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:43:47.523572 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:47.523525 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" podStartSLOduration=2.200442158 podStartE2EDuration="4.523512638s" totalImagesPullingTime="2.32307048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:43:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:43:47.522358219 +0000 UTC m=+343.673255251" watchObservedRunningTime="2026-10-30 15:43:47.523512638 +0000 UTC m=+343.674409668" Oct 30 15:43:53.794499 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:43:53.794463 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:43:56.975375 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:43:56.975337 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:44:08.477931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:08.477903 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-q55hh" Oct 30 15:44:26.976283 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:44:26.976246 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:44:32.627621 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:32.627581 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:44:32.878099 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:32.878013 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-c7x2d"] Oct 30 15:44:32.886394 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:32.886361 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:44:32.891500 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:32.891475 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-c7x2d" Oct 30 15:44:32.910877 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:32.910852 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-c7x2d"] Oct 30 15:44:32.934283 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:32.934257 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh"] Oct 30 15:44:32.937650 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:32.937634 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" Oct 30 15:44:32.988115 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:32.988093 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-ckjd6\"" Oct 30 15:44:32.988311 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:32.988140 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Oct 30 15:44:32.988311 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:32.988119 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-k46zv\"" Oct 30 15:44:32.988478 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:32.988160 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Oct 30 15:44:32.989507 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:32.989491 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh"] Oct 30 15:44:32.999673 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:32.999648 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6h8h\" (UniqueName: \"kubernetes.io/projected/e866b86c-ac64-45a1-af1d-5e6d59aeaa52-kube-api-access-p6h8h\") pod \"kserve-controller-manager-776cc66d47-c7x2d\" (UID: \"e866b86c-ac64-45a1-af1d-5e6d59aeaa52\") " pod="opendatahub/kserve-controller-manager-776cc66d47-c7x2d" Oct 30 15:44:32.999782 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:32.999690 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e866b86c-ac64-45a1-af1d-5e6d59aeaa52-cert\") pod \"kserve-controller-manager-776cc66d47-c7x2d\" (UID: \"e866b86c-ac64-45a1-af1d-5e6d59aeaa52\") " pod="opendatahub/kserve-controller-manager-776cc66d47-c7x2d" Oct 30 15:44:33.067222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.067179 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-pxchz"] Oct 30 15:44:33.070584 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.070565 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" Oct 30 15:44:33.088240 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.088218 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-pxchz"] Oct 30 15:44:33.088356 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.088255 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:44:33.100659 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.100638 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6h8h\" (UniqueName: \"kubernetes.io/projected/e866b86c-ac64-45a1-af1d-5e6d59aeaa52-kube-api-access-p6h8h\") pod \"kserve-controller-manager-776cc66d47-c7x2d\" (UID: \"e866b86c-ac64-45a1-af1d-5e6d59aeaa52\") " pod="opendatahub/kserve-controller-manager-776cc66d47-c7x2d" Oct 30 15:44:33.100767 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.100675 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fdb844e1-85f8-4d39-97b1-bfd946ef4d34-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-9g6sh\" (UID: \"fdb844e1-85f8-4d39-97b1-bfd946ef4d34\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" Oct 30 15:44:33.100767 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.100695 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e866b86c-ac64-45a1-af1d-5e6d59aeaa52-cert\") pod \"kserve-controller-manager-776cc66d47-c7x2d\" (UID: \"e866b86c-ac64-45a1-af1d-5e6d59aeaa52\") " pod="opendatahub/kserve-controller-manager-776cc66d47-c7x2d" Oct 30 15:44:33.100888 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.100779 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sf22\" (UniqueName: \"kubernetes.io/projected/fdb844e1-85f8-4d39-97b1-bfd946ef4d34-kube-api-access-6sf22\") pod \"llmisvc-controller-manager-869d5dcc5d-9g6sh\" (UID: \"fdb844e1-85f8-4d39-97b1-bfd946ef4d34\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" Oct 30 15:44:33.103220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.103170 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e866b86c-ac64-45a1-af1d-5e6d59aeaa52-cert\") pod \"kserve-controller-manager-776cc66d47-c7x2d\" (UID: \"e866b86c-ac64-45a1-af1d-5e6d59aeaa52\") " pod="opendatahub/kserve-controller-manager-776cc66d47-c7x2d" Oct 30 15:44:33.104832 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.104814 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-kfk62\"" Oct 30 15:44:33.104922 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.104876 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\"" Oct 30 15:44:33.104991 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.104976 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Oct 30 15:44:33.124119 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.124096 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6h8h\" (UniqueName: \"kubernetes.io/projected/e866b86c-ac64-45a1-af1d-5e6d59aeaa52-kube-api-access-p6h8h\") pod \"kserve-controller-manager-776cc66d47-c7x2d\" (UID: \"e866b86c-ac64-45a1-af1d-5e6d59aeaa52\") " pod="opendatahub/kserve-controller-manager-776cc66d47-c7x2d" Oct 30 15:44:33.202099 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.202027 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95519221-ff43-4b09-9ec5-cf98e595d03e-cert\") pod \"odh-model-controller-f87fcf56c-pxchz\" (UID: \"95519221-ff43-4b09-9ec5-cf98e595d03e\") " pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" Oct 30 15:44:33.202099 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.202066 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4slb\" (UniqueName: \"kubernetes.io/projected/95519221-ff43-4b09-9ec5-cf98e595d03e-kube-api-access-s4slb\") pod \"odh-model-controller-f87fcf56c-pxchz\" (UID: \"95519221-ff43-4b09-9ec5-cf98e595d03e\") " pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" Oct 30 15:44:33.202099 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.202095 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fdb844e1-85f8-4d39-97b1-bfd946ef4d34-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-9g6sh\" (UID: \"fdb844e1-85f8-4d39-97b1-bfd946ef4d34\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" Oct 30 15:44:33.202370 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.202314 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95519221-ff43-4b09-9ec5-cf98e595d03e-metrics-certs\") pod \"odh-model-controller-f87fcf56c-pxchz\" (UID: \"95519221-ff43-4b09-9ec5-cf98e595d03e\") " pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" Oct 30 15:44:33.202370 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.202334 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sf22\" (UniqueName: \"kubernetes.io/projected/fdb844e1-85f8-4d39-97b1-bfd946ef4d34-kube-api-access-6sf22\") pod \"llmisvc-controller-manager-869d5dcc5d-9g6sh\" (UID: \"fdb844e1-85f8-4d39-97b1-bfd946ef4d34\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" Oct 30 15:44:33.204594 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.204561 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fdb844e1-85f8-4d39-97b1-bfd946ef4d34-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-9g6sh\" (UID: \"fdb844e1-85f8-4d39-97b1-bfd946ef4d34\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" Oct 30 15:44:33.206417 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.206390 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-c7x2d" Oct 30 15:44:33.225595 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.225571 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sf22\" (UniqueName: \"kubernetes.io/projected/fdb844e1-85f8-4d39-97b1-bfd946ef4d34-kube-api-access-6sf22\") pod \"llmisvc-controller-manager-869d5dcc5d-9g6sh\" (UID: \"fdb844e1-85f8-4d39-97b1-bfd946ef4d34\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" Oct 30 15:44:33.249048 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.249017 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" Oct 30 15:44:33.267447 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.267423 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:44:33.303069 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.303035 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95519221-ff43-4b09-9ec5-cf98e595d03e-metrics-certs\") pod \"odh-model-controller-f87fcf56c-pxchz\" (UID: \"95519221-ff43-4b09-9ec5-cf98e595d03e\") " pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" Oct 30 15:44:33.303260 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.303083 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95519221-ff43-4b09-9ec5-cf98e595d03e-cert\") pod \"odh-model-controller-f87fcf56c-pxchz\" (UID: \"95519221-ff43-4b09-9ec5-cf98e595d03e\") " pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" Oct 30 15:44:33.303260 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.303122 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4slb\" (UniqueName: \"kubernetes.io/projected/95519221-ff43-4b09-9ec5-cf98e595d03e-kube-api-access-s4slb\") pod \"odh-model-controller-f87fcf56c-pxchz\" (UID: \"95519221-ff43-4b09-9ec5-cf98e595d03e\") " pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" Oct 30 15:44:33.306230 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.305787 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95519221-ff43-4b09-9ec5-cf98e595d03e-metrics-certs\") pod \"odh-model-controller-f87fcf56c-pxchz\" (UID: \"95519221-ff43-4b09-9ec5-cf98e595d03e\") " pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" Oct 30 15:44:33.306999 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.306979 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95519221-ff43-4b09-9ec5-cf98e595d03e-cert\") pod \"odh-model-controller-f87fcf56c-pxchz\" (UID: \"95519221-ff43-4b09-9ec5-cf98e595d03e\") " pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" Oct 30 15:44:33.326011 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.325987 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4slb\" (UniqueName: \"kubernetes.io/projected/95519221-ff43-4b09-9ec5-cf98e595d03e-kube-api-access-s4slb\") pod \"odh-model-controller-f87fcf56c-pxchz\" (UID: \"95519221-ff43-4b09-9ec5-cf98e595d03e\") " pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" Oct 30 15:44:33.382611 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.382581 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" Oct 30 15:44:33.508566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.508537 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:44:33.509940 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.509915 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-c7x2d"] Oct 30 15:44:33.511006 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:44:33.510981 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode866b86c_ac64_45a1_af1d_5e6d59aeaa52.slice/crio-724816df3a28d17111a5fe73924a00a38516ec302b168bc8b66a4f51bc150332 WatchSource:0}: Error finding container 724816df3a28d17111a5fe73924a00a38516ec302b168bc8b66a4f51bc150332: Status 404 returned error can't find the container with id 724816df3a28d17111a5fe73924a00a38516ec302b168bc8b66a4f51bc150332 Oct 30 15:44:33.556026 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.556005 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh"] Oct 30 15:44:33.557149 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:44:33.557126 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfdb844e1_85f8_4d39_97b1_bfd946ef4d34.slice/crio-43b03fa9886c427be5933f767082aeecc58f381162f26c64525d1811b9cea88c WatchSource:0}: Error finding container 43b03fa9886c427be5933f767082aeecc58f381162f26c64525d1811b9cea88c: Status 404 returned error can't find the container with id 43b03fa9886c427be5933f767082aeecc58f381162f26c64525d1811b9cea88c Oct 30 15:44:33.654946 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.654909 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" event={"ID":"fdb844e1-85f8-4d39-97b1-bfd946ef4d34","Type":"ContainerStarted","Data":"43b03fa9886c427be5933f767082aeecc58f381162f26c64525d1811b9cea88c"} Oct 30 15:44:33.656231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.656207 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-c7x2d" event={"ID":"e866b86c-ac64-45a1-af1d-5e6d59aeaa52","Type":"ContainerStarted","Data":"724816df3a28d17111a5fe73924a00a38516ec302b168bc8b66a4f51bc150332"} Oct 30 15:44:33.690956 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:33.690935 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-pxchz"] Oct 30 15:44:33.692141 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:44:33.692120 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95519221_ff43_4b09_9ec5_cf98e595d03e.slice/crio-ad543338594e7c28ba7531eedd8c21fbf8385a333289999f80537ef9f7900e23 WatchSource:0}: Error finding container ad543338594e7c28ba7531eedd8c21fbf8385a333289999f80537ef9f7900e23: Status 404 returned error can't find the container with id ad543338594e7c28ba7531eedd8c21fbf8385a333289999f80537ef9f7900e23 Oct 30 15:44:34.665839 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:34.665750 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" event={"ID":"95519221-ff43-4b09-9ec5-cf98e595d03e","Type":"ContainerStarted","Data":"ad543338594e7c28ba7531eedd8c21fbf8385a333289999f80537ef9f7900e23"} Oct 30 15:44:35.667654 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:35.667622 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:44:35.828322 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:35.828237 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:44:35.994054 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:35.993915 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:44:36.159981 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:36.159953 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:44:36.682327 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:36.682290 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" event={"ID":"fdb844e1-85f8-4d39-97b1-bfd946ef4d34","Type":"ContainerStarted","Data":"9852920792e00152c7e5437e48a1d55a919c2f86a263bb45ef0156dcc971c6cf"} Oct 30 15:44:36.682786 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:36.682502 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" Oct 30 15:44:36.683986 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:36.683956 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-c7x2d" event={"ID":"e866b86c-ac64-45a1-af1d-5e6d59aeaa52","Type":"ContainerStarted","Data":"8b4e03bc9f0830d971fd37359cc6d5e127f0058659cb3f123e203c902f7c6321"} Oct 30 15:44:36.684097 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:36.684041 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-c7x2d" Oct 30 15:44:36.726461 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:36.726422 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" podStartSLOduration=2.098380412 podStartE2EDuration="4.726408981s" totalImagesPullingTime="2.628028569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:44:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:44:36.725266268 +0000 UTC m=+392.876163299" watchObservedRunningTime="2026-10-30 15:44:36.726408981 +0000 UTC m=+392.877306014" Oct 30 15:44:36.765577 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:36.765535 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-c7x2d" podStartSLOduration=2.081534903 podStartE2EDuration="4.76552172s" totalImagesPullingTime="2.683986817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:44:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:44:36.764487362 +0000 UTC m=+392.915384394" watchObservedRunningTime="2026-10-30 15:44:36.76552172 +0000 UTC m=+392.916418750" Oct 30 15:44:37.690116 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:37.690079 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" event={"ID":"95519221-ff43-4b09-9ec5-cf98e595d03e","Type":"ContainerStarted","Data":"73b4cbf2b0c75450ba6315b8fc867623e97e10541804c6df6ce0ce773637493d"} Oct 30 15:44:37.690659 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:37.690230 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" Oct 30 15:44:37.729791 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:37.729743 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" podStartSLOduration=1.739100649 podStartE2EDuration="4.729730756s" totalImagesPullingTime="2.990630107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:44:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:44:37.729240217 +0000 UTC m=+393.880137252" watchObservedRunningTime="2026-10-30 15:44:37.729730756 +0000 UTC m=+393.880627787" Oct 30 15:44:48.697562 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:48.697486 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-f87fcf56c-pxchz" Oct 30 15:44:53.916907 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:44:53.916877 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:44:56.975603 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:44:56.975568 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:45:07.692497 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:07.692462 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" Oct 30 15:45:07.695738 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:07.695716 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-c7x2d" Oct 30 15:45:20.440367 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:20.440331 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:20.617637 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:20.617601 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:26.976345 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:45:26.976308 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:45:31.574731 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:31.574700 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:32.380476 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:32.380432 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:35.658014 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:35.657981 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:36.017550 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:36.017458 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:37.790166 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:37.790133 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:38.026625 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:38.026593 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:39.279101 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:39.279068 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-kvlxj"] Oct 30 15:45:39.281932 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:39.281911 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-kvlxj" Oct 30 15:45:39.310053 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:39.310025 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-xp5xn\"" Oct 30 15:45:39.310053 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:39.310052 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Oct 30 15:45:39.310442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:39.310067 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Oct 30 15:45:39.310442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:39.310089 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Oct 30 15:45:39.311813 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:39.311794 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-kvlxj"] Oct 30 15:45:39.416282 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:39.416253 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf2nq\" (UniqueName: \"kubernetes.io/projected/350b80bd-337e-4eeb-a491-bcd3e47834cf-kube-api-access-nf2nq\") pod \"dns-operator-controller-manager-65bfd79598-kvlxj\" (UID: \"350b80bd-337e-4eeb-a491-bcd3e47834cf\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-kvlxj" Oct 30 15:45:39.517150 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:39.517122 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf2nq\" (UniqueName: \"kubernetes.io/projected/350b80bd-337e-4eeb-a491-bcd3e47834cf-kube-api-access-nf2nq\") pod \"dns-operator-controller-manager-65bfd79598-kvlxj\" (UID: \"350b80bd-337e-4eeb-a491-bcd3e47834cf\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-kvlxj" Oct 30 15:45:39.555889 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:39.555824 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf2nq\" (UniqueName: \"kubernetes.io/projected/350b80bd-337e-4eeb-a491-bcd3e47834cf-kube-api-access-nf2nq\") pod \"dns-operator-controller-manager-65bfd79598-kvlxj\" (UID: \"350b80bd-337e-4eeb-a491-bcd3e47834cf\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-kvlxj" Oct 30 15:45:39.593678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:39.593655 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-kvlxj" Oct 30 15:45:39.801257 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:39.801228 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:39.910406 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:39.910385 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-kvlxj"] Oct 30 15:45:39.912465 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:45:39.912437 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod350b80bd_337e_4eeb_a491_bcd3e47834cf.slice/crio-c6daac96f08759239915e4d4879b91967713008496225ace440e2357623bf6f7 WatchSource:0}: Error finding container c6daac96f08759239915e4d4879b91967713008496225ace440e2357623bf6f7: Status 404 returned error can't find the container with id c6daac96f08759239915e4d4879b91967713008496225ace440e2357623bf6f7 Oct 30 15:45:39.956575 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:39.956547 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-kvlxj" event={"ID":"350b80bd-337e-4eeb-a491-bcd3e47834cf","Type":"ContainerStarted","Data":"c6daac96f08759239915e4d4879b91967713008496225ace440e2357623bf6f7"} Oct 30 15:45:40.186224 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:40.186126 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:40.552449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:40.552378 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:40.747556 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:40.747519 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:41.795627 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:41.795587 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:42.977733 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:42.977690 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-kvlxj" event={"ID":"350b80bd-337e-4eeb-a491-bcd3e47834cf","Type":"ContainerStarted","Data":"937fb27404cbe4c604e2fa1ddd0c2c76b2f904a576816ee77095572efaa00c66"} Oct 30 15:45:42.978188 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:42.977752 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-kvlxj" Oct 30 15:45:43.050405 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:43.050352 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-kvlxj" podStartSLOduration=1.919916072 podStartE2EDuration="4.050338963s" totalImagesPullingTime="2.130422891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:45:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:45:43.049387531 +0000 UTC m=+459.200284562" watchObservedRunningTime="2026-10-30 15:45:43.050338963 +0000 UTC m=+459.201235993" Oct 30 15:45:45.596319 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:45.596275 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:45.773657 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:45.773620 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:46.216765 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:46.216734 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:46.402316 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:46.402282 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:48.639723 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:48.639692 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:48.837525 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:48.837492 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:49.295512 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:49.295487 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:49.476499 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:49.476464 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:51.270394 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:51.270352 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:51.474028 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:51.473994 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:51.814021 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:51.813985 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9"] Oct 30 15:45:51.817418 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:51.817403 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9" Oct 30 15:45:51.846949 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:51.846925 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Oct 30 15:45:51.847114 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:51.846961 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-b9tr5\"" Oct 30 15:45:51.847114 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:51.846977 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Oct 30 15:45:51.848380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:51.848362 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9"] Oct 30 15:45:51.866573 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:51.866547 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:51.922960 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:51.922938 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bf99dd84-8640-43ee-8fa8-acb26616da8c-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-tjbk9\" (UID: \"bf99dd84-8640-43ee-8fa8-acb26616da8c\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9" Oct 30 15:45:51.923091 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:51.922977 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf99dd84-8640-43ee-8fa8-acb26616da8c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-tjbk9\" (UID: \"bf99dd84-8640-43ee-8fa8-acb26616da8c\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9" Oct 30 15:45:51.923091 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:51.923067 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8xgx\" (UniqueName: \"kubernetes.io/projected/bf99dd84-8640-43ee-8fa8-acb26616da8c-kube-api-access-z8xgx\") pod \"kuadrant-console-plugin-66c4fb575d-tjbk9\" (UID: \"bf99dd84-8640-43ee-8fa8-acb26616da8c\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9" Oct 30 15:45:52.024347 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:52.024324 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf99dd84-8640-43ee-8fa8-acb26616da8c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-tjbk9\" (UID: \"bf99dd84-8640-43ee-8fa8-acb26616da8c\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9" Oct 30 15:45:52.024484 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:52.024368 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8xgx\" (UniqueName: \"kubernetes.io/projected/bf99dd84-8640-43ee-8fa8-acb26616da8c-kube-api-access-z8xgx\") pod \"kuadrant-console-plugin-66c4fb575d-tjbk9\" (UID: \"bf99dd84-8640-43ee-8fa8-acb26616da8c\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9" Oct 30 15:45:52.024484 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:52.024411 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bf99dd84-8640-43ee-8fa8-acb26616da8c-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-tjbk9\" (UID: \"bf99dd84-8640-43ee-8fa8-acb26616da8c\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9" Oct 30 15:45:52.024960 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:52.024942 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bf99dd84-8640-43ee-8fa8-acb26616da8c-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-tjbk9\" (UID: \"bf99dd84-8640-43ee-8fa8-acb26616da8c\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9" Oct 30 15:45:52.026927 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:52.026899 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf99dd84-8640-43ee-8fa8-acb26616da8c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-tjbk9\" (UID: \"bf99dd84-8640-43ee-8fa8-acb26616da8c\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9" Oct 30 15:45:52.036089 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:52.036047 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:52.048288 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:52.048267 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8xgx\" (UniqueName: \"kubernetes.io/projected/bf99dd84-8640-43ee-8fa8-acb26616da8c-kube-api-access-z8xgx\") pod \"kuadrant-console-plugin-66c4fb575d-tjbk9\" (UID: \"bf99dd84-8640-43ee-8fa8-acb26616da8c\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9" Oct 30 15:45:52.129466 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:52.129445 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9" Oct 30 15:45:52.425317 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:52.425290 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9"] Oct 30 15:45:52.426767 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:45:52.426745 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf99dd84_8640_43ee_8fa8_acb26616da8c.slice/crio-e25c49be6690a352c9c9a7c361211aef541fbc73899b61641688690c03dc2775 WatchSource:0}: Error finding container e25c49be6690a352c9c9a7c361211aef541fbc73899b61641688690c03dc2775: Status 404 returned error can't find the container with id e25c49be6690a352c9c9a7c361211aef541fbc73899b61641688690c03dc2775 Oct 30 15:45:53.021673 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:53.021645 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9" event={"ID":"bf99dd84-8640-43ee-8fa8-acb26616da8c","Type":"ContainerStarted","Data":"e25c49be6690a352c9c9a7c361211aef541fbc73899b61641688690c03dc2775"} Oct 30 15:45:53.987680 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:53.987649 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-kvlxj" Oct 30 15:45:54.086088 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:54.086059 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:56.974720 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:45:56.974684 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:45:57.039586 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:57.039553 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9" event={"ID":"bf99dd84-8640-43ee-8fa8-acb26616da8c","Type":"ContainerStarted","Data":"6feaa68c16eb84bb4d4b9d8e21e24db88365065f8287b808a0e63cfb88ac7d86"} Oct 30 15:45:57.078809 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:57.078764 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-tjbk9" podStartSLOduration=1.65548528 podStartE2EDuration="6.07875129s" totalImagesPullingTime="4.42326601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:45:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:45:57.077993849 +0000 UTC m=+473.228890883" watchObservedRunningTime="2026-10-30 15:45:57.07875129 +0000 UTC m=+473.229648320" Oct 30 15:45:57.605726 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:57.605696 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:57.773576 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:57.773545 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:59.173543 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:59.173506 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:45:59.985469 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:45:59.985437 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:04.883268 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:04.883238 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:05.427892 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:05.427860 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:08.504395 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:08.504359 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:08.704602 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:08.704557 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:11.282498 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:11.282466 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:11.448622 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:11.448589 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:16.938253 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:16.938161 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:17.161075 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:17.161034 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:17.357528 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:17.357448 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:26.975528 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:46:26.975490 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:46:36.074720 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:36.074681 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:36.236128 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:36.236100 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:50.620548 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:50.620513 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-wt7xn"] Oct 30 15:46:50.623873 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:50.623853 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" Oct 30 15:46:50.640695 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:50.640671 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Oct 30 15:46:50.641933 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:50.641915 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-wt7xn"] Oct 30 15:46:50.671660 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:50.671637 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:50.747703 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:50.747672 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-wt7xn"] Oct 30 15:46:50.793304 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:50.793269 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m48w2\" (UniqueName: \"kubernetes.io/projected/0492d9c7-e1c1-44dc-9acf-376d53daa0f3-kube-api-access-m48w2\") pod \"limitador-limitador-74dfcfb894-wt7xn\" (UID: \"0492d9c7-e1c1-44dc-9acf-376d53daa0f3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" Oct 30 15:46:50.793457 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:50.793371 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0492d9c7-e1c1-44dc-9acf-376d53daa0f3-config-file\") pod \"limitador-limitador-74dfcfb894-wt7xn\" (UID: \"0492d9c7-e1c1-44dc-9acf-376d53daa0f3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" Oct 30 15:46:50.841618 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:50.841593 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:50.894652 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:50.894628 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m48w2\" (UniqueName: \"kubernetes.io/projected/0492d9c7-e1c1-44dc-9acf-376d53daa0f3-kube-api-access-m48w2\") pod \"limitador-limitador-74dfcfb894-wt7xn\" (UID: \"0492d9c7-e1c1-44dc-9acf-376d53daa0f3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" Oct 30 15:46:50.894753 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:50.894663 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0492d9c7-e1c1-44dc-9acf-376d53daa0f3-config-file\") pod \"limitador-limitador-74dfcfb894-wt7xn\" (UID: \"0492d9c7-e1c1-44dc-9acf-376d53daa0f3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" Oct 30 15:46:50.895338 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:50.895321 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0492d9c7-e1c1-44dc-9acf-376d53daa0f3-config-file\") pod \"limitador-limitador-74dfcfb894-wt7xn\" (UID: \"0492d9c7-e1c1-44dc-9acf-376d53daa0f3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" Oct 30 15:46:50.918211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:50.918177 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m48w2\" (UniqueName: \"kubernetes.io/projected/0492d9c7-e1c1-44dc-9acf-376d53daa0f3-kube-api-access-m48w2\") pod \"limitador-limitador-74dfcfb894-wt7xn\" (UID: \"0492d9c7-e1c1-44dc-9acf-376d53daa0f3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" Oct 30 15:46:50.936243 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:50.936223 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" Oct 30 15:46:51.013013 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:51.012985 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:51.231231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:51.231206 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-wt7xn"] Oct 30 15:46:51.232561 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:46:51.232535 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0492d9c7_e1c1_44dc_9acf_376d53daa0f3.slice/crio-cc754886a0cc1d27738348c6ccf07196adfbcbf5c7c84b88465ab446c878345e WatchSource:0}: Error finding container cc754886a0cc1d27738348c6ccf07196adfbcbf5c7c84b88465ab446c878345e: Status 404 returned error can't find the container with id cc754886a0cc1d27738348c6ccf07196adfbcbf5c7c84b88465ab446c878345e Oct 30 15:46:51.263267 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:51.263239 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" event={"ID":"0492d9c7-e1c1-44dc-9acf-376d53daa0f3","Type":"ContainerStarted","Data":"cc754886a0cc1d27738348c6ccf07196adfbcbf5c7c84b88465ab446c878345e"} Oct 30 15:46:51.642721 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:51.642694 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:51.810276 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:51.810244 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:53.272910 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:53.272874 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" event={"ID":"0492d9c7-e1c1-44dc-9acf-376d53daa0f3","Type":"ContainerStarted","Data":"c71e3c81f6fd1b9460bcccd4b1a9fe8d3d30f6a76d7f73fb508d8475758e14f2"} Oct 30 15:46:53.273419 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:53.273008 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" Oct 30 15:46:53.313277 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:53.313233 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" podStartSLOduration=2.283406451 podStartE2EDuration="3.313223078s" totalImagesPullingTime="1.029816627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:46:53.312510465 +0000 UTC m=+529.463407497" watchObservedRunningTime="2026-10-30 15:46:53.313223078 +0000 UTC m=+529.464120106" Oct 30 15:46:54.181330 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:54.181295 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:54.352080 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:54.352050 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:56.975824 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:46:56.975793 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:46:59.472135 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:59.472101 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:46:59.881169 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:46:59.881140 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:04.277331 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:04.277305 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" Oct 30 15:47:04.918619 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:04.918591 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-wt7xn"] Oct 30 15:47:04.918823 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:04.918791 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" podUID="0492d9c7-e1c1-44dc-9acf-376d53daa0f3" containerName="limitador" containerID="cri-o://c71e3c81f6fd1b9460bcccd4b1a9fe8d3d30f6a76d7f73fb508d8475758e14f2" gracePeriod=30 Oct 30 15:47:05.469989 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:05.469967 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" Oct 30 15:47:05.513149 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:05.513077 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0492d9c7-e1c1-44dc-9acf-376d53daa0f3-config-file\" (UniqueName: \"kubernetes.io/configmap/0492d9c7-e1c1-44dc-9acf-376d53daa0f3-config-file\") pod \"0492d9c7-e1c1-44dc-9acf-376d53daa0f3\" (UID: \"0492d9c7-e1c1-44dc-9acf-376d53daa0f3\") " Oct 30 15:47:05.513294 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:05.513152 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0492d9c7-e1c1-44dc-9acf-376d53daa0f3-kube-api-access-m48w2\" (UniqueName: \"kubernetes.io/projected/0492d9c7-e1c1-44dc-9acf-376d53daa0f3-kube-api-access-m48w2\") pod \"0492d9c7-e1c1-44dc-9acf-376d53daa0f3\" (UID: \"0492d9c7-e1c1-44dc-9acf-376d53daa0f3\") " Oct 30 15:47:05.513483 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:05.513455 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0492d9c7-e1c1-44dc-9acf-376d53daa0f3-config-file" pod "0492d9c7-e1c1-44dc-9acf-376d53daa0f3" (UID: "0492d9c7-e1c1-44dc-9acf-376d53daa0f3"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:47:05.515650 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:05.515622 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0492d9c7-e1c1-44dc-9acf-376d53daa0f3-kube-api-access-m48w2" pod "0492d9c7-e1c1-44dc-9acf-376d53daa0f3" (UID: "0492d9c7-e1c1-44dc-9acf-376d53daa0f3"). InnerVolumeSpecName "kube-api-access-m48w2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:47:05.614050 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:05.614015 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m48w2\" (UniqueName: \"kubernetes.io/projected/0492d9c7-e1c1-44dc-9acf-376d53daa0f3-kube-api-access-m48w2\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:47:05.614050 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:05.614047 5013 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0492d9c7-e1c1-44dc-9acf-376d53daa0f3-config-file\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:47:06.329287 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:06.329176 5013 generic.go:356] "Generic (PLEG): container finished" podID="0492d9c7-e1c1-44dc-9acf-376d53daa0f3" containerID="c71e3c81f6fd1b9460bcccd4b1a9fe8d3d30f6a76d7f73fb508d8475758e14f2" exitCode=0 Oct 30 15:47:06.329287 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:06.329261 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" event={"ID":"0492d9c7-e1c1-44dc-9acf-376d53daa0f3","Type":"ContainerDied","Data":"c71e3c81f6fd1b9460bcccd4b1a9fe8d3d30f6a76d7f73fb508d8475758e14f2"} Oct 30 15:47:06.329505 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:06.329309 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" event={"ID":"0492d9c7-e1c1-44dc-9acf-376d53daa0f3","Type":"ContainerDied","Data":"cc754886a0cc1d27738348c6ccf07196adfbcbf5c7c84b88465ab446c878345e"} Oct 30 15:47:06.329505 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:06.329329 5013 scope.go:122] "RemoveContainer" containerID="c71e3c81f6fd1b9460bcccd4b1a9fe8d3d30f6a76d7f73fb508d8475758e14f2" Oct 30 15:47:06.329505 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:06.329265 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-wt7xn" Oct 30 15:47:06.339601 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:06.339576 5013 scope.go:122] "RemoveContainer" containerID="c71e3c81f6fd1b9460bcccd4b1a9fe8d3d30f6a76d7f73fb508d8475758e14f2" Oct 30 15:47:06.339875 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:47:06.339853 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c71e3c81f6fd1b9460bcccd4b1a9fe8d3d30f6a76d7f73fb508d8475758e14f2\": container with ID starting with c71e3c81f6fd1b9460bcccd4b1a9fe8d3d30f6a76d7f73fb508d8475758e14f2 not found: ID does not exist" containerID="c71e3c81f6fd1b9460bcccd4b1a9fe8d3d30f6a76d7f73fb508d8475758e14f2" Oct 30 15:47:06.339941 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:06.339889 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c71e3c81f6fd1b9460bcccd4b1a9fe8d3d30f6a76d7f73fb508d8475758e14f2"} err="failed to get container status \"c71e3c81f6fd1b9460bcccd4b1a9fe8d3d30f6a76d7f73fb508d8475758e14f2\": rpc error: code = NotFound desc = could not find container \"c71e3c81f6fd1b9460bcccd4b1a9fe8d3d30f6a76d7f73fb508d8475758e14f2\": container with ID starting with c71e3c81f6fd1b9460bcccd4b1a9fe8d3d30f6a76d7f73fb508d8475758e14f2 not found: ID does not exist" Oct 30 15:47:06.378944 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:06.378917 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-wt7xn"] Oct 30 15:47:06.390466 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:06.390440 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-wt7xn"] Oct 30 15:47:08.003913 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:08.003880 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0492d9c7-e1c1-44dc-9acf-376d53daa0f3" path="/var/lib/kubelet/pods/0492d9c7-e1c1-44dc-9acf-376d53daa0f3/volumes" Oct 30 15:47:10.422443 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:10.422414 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:10.594984 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:10.594953 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:11.083550 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.083521 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn"] Oct 30 15:47:11.083951 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.083934 5013 cpu_manager.go:414] "containerMap: removing container" podUID="0492d9c7-e1c1-44dc-9acf-376d53daa0f3" containerName="limitador" Oct 30 15:47:11.084041 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.083954 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0492d9c7-e1c1-44dc-9acf-376d53daa0f3" containerName="limitador" Oct 30 15:47:11.084102 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.084088 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="0492d9c7-e1c1-44dc-9acf-376d53daa0f3" containerName="limitador" Oct 30 15:47:11.088477 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.088458 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn" Oct 30 15:47:11.105691 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.105671 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn"] Oct 30 15:47:11.115271 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.115249 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 15:47:11.115271 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.115249 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-4lhqg\"" Oct 30 15:47:11.115271 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.115252 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 15:47:11.115453 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.115259 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Oct 30 15:47:11.157731 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.157708 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/20477300-291f-4372-a44a-69547e1b7216-data\") pod \"postgres-7cdbf6dc44-tlqhn\" (UID: \"20477300-291f-4372-a44a-69547e1b7216\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn" Oct 30 15:47:11.157839 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.157741 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ldv4\" (UniqueName: \"kubernetes.io/projected/20477300-291f-4372-a44a-69547e1b7216-kube-api-access-7ldv4\") pod \"postgres-7cdbf6dc44-tlqhn\" (UID: \"20477300-291f-4372-a44a-69547e1b7216\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn" Oct 30 15:47:11.218388 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.218361 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:11.258589 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.258566 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/20477300-291f-4372-a44a-69547e1b7216-data\") pod \"postgres-7cdbf6dc44-tlqhn\" (UID: \"20477300-291f-4372-a44a-69547e1b7216\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn" Oct 30 15:47:11.258664 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.258595 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ldv4\" (UniqueName: \"kubernetes.io/projected/20477300-291f-4372-a44a-69547e1b7216-kube-api-access-7ldv4\") pod \"postgres-7cdbf6dc44-tlqhn\" (UID: \"20477300-291f-4372-a44a-69547e1b7216\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn" Oct 30 15:47:11.258923 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.258905 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/20477300-291f-4372-a44a-69547e1b7216-data\") pod \"postgres-7cdbf6dc44-tlqhn\" (UID: \"20477300-291f-4372-a44a-69547e1b7216\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn" Oct 30 15:47:11.281467 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.281442 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ldv4\" (UniqueName: \"kubernetes.io/projected/20477300-291f-4372-a44a-69547e1b7216-kube-api-access-7ldv4\") pod \"postgres-7cdbf6dc44-tlqhn\" (UID: \"20477300-291f-4372-a44a-69547e1b7216\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn" Oct 30 15:47:11.401288 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.401265 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn" Oct 30 15:47:11.697350 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:11.697318 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn"] Oct 30 15:47:11.698143 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:47:11.698119 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20477300_291f_4372_a44a_69547e1b7216.slice/crio-822a249fd4dbf2bc9a7c57310dfec85b3abf9056a866c0eb4561c9659ca3e6c1 WatchSource:0}: Error finding container 822a249fd4dbf2bc9a7c57310dfec85b3abf9056a866c0eb4561c9659ca3e6c1: Status 404 returned error can't find the container with id 822a249fd4dbf2bc9a7c57310dfec85b3abf9056a866c0eb4561c9659ca3e6c1 Oct 30 15:47:12.361716 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:12.361606 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn" event={"ID":"20477300-291f-4372-a44a-69547e1b7216","Type":"ContainerStarted","Data":"822a249fd4dbf2bc9a7c57310dfec85b3abf9056a866c0eb4561c9659ca3e6c1"} Oct 30 15:47:16.188631 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:16.188575 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:17.391026 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:17.390990 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn" event={"ID":"20477300-291f-4372-a44a-69547e1b7216","Type":"ContainerStarted","Data":"d53a4a5eb779bc02b1376a5c216ed5063396e106d465136fd3de04f93e389d97"} Oct 30 15:47:17.391421 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:17.391106 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn" Oct 30 15:47:17.431272 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:17.431220 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn" podStartSLOduration=1.800434127 podStartE2EDuration="6.431189359s" totalImagesPullingTime="4.630755232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:47:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:47:17.430040001 +0000 UTC m=+553.580937032" watchObservedRunningTime="2026-10-30 15:47:17.431189359 +0000 UTC m=+553.582086389" Oct 30 15:47:23.426226 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:23.426180 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-tlqhn" Oct 30 15:47:23.959481 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:23.959401 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:24.131461 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:24.131430 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:25.416375 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:25.416343 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:26.976746 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:47:26.976706 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:47:34.501646 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.501587 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:34.514879 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.514843 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-577f596567-qd7qg"] Oct 30 15:47:34.531414 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.531391 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:34.540339 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.540313 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-qd7qg"] Oct 30 15:47:34.550372 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.550342 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Oct 30 15:47:34.550523 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.550510 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lcl2r\"" Oct 30 15:47:34.550602 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.550562 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Oct 30 15:47:34.550602 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.550579 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Oct 30 15:47:34.654779 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.654748 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-webhook-cert\") pod \"maas-controller-577f596567-qd7qg\" (UID: \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\") " pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:34.654948 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.654798 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnqkm\" (UniqueName: \"kubernetes.io/projected/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-kube-api-access-gnqkm\") pod \"maas-controller-577f596567-qd7qg\" (UID: \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\") " pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:34.654948 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.654896 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-metrics-certs\") pod \"maas-controller-577f596567-qd7qg\" (UID: \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\") " pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:34.755589 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.755512 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-webhook-cert\") pod \"maas-controller-577f596567-qd7qg\" (UID: \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\") " pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:34.755589 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.755573 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnqkm\" (UniqueName: \"kubernetes.io/projected/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-kube-api-access-gnqkm\") pod \"maas-controller-577f596567-qd7qg\" (UID: \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\") " pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:34.755757 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.755592 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-metrics-certs\") pod \"maas-controller-577f596567-qd7qg\" (UID: \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\") " pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:34.758137 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.758110 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-metrics-certs\") pod \"maas-controller-577f596567-qd7qg\" (UID: \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\") " pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:34.758270 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.758227 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-webhook-cert\") pod \"maas-controller-577f596567-qd7qg\" (UID: \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\") " pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:34.778602 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.778579 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnqkm\" (UniqueName: \"kubernetes.io/projected/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-kube-api-access-gnqkm\") pod \"maas-controller-577f596567-qd7qg\" (UID: \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\") " pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:34.845593 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.845558 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:34.932535 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.932501 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5964675dcd-lswsg"] Oct 30 15:47:34.940481 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.940458 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 15:47:34.955730 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:34.955707 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5964675dcd-lswsg"] Oct 30 15:47:35.058683 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:35.058607 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44zf9\" (UniqueName: \"kubernetes.io/projected/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-kube-api-access-44zf9\") pod \"maas-controller-5964675dcd-lswsg\" (UID: \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\") " pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 15:47:35.058848 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:35.058772 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-metrics-certs\") pod \"maas-controller-5964675dcd-lswsg\" (UID: \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\") " pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 15:47:35.058848 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:35.058803 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-webhook-cert\") pod \"maas-controller-5964675dcd-lswsg\" (UID: \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\") " pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 15:47:35.141987 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:35.141960 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-qd7qg"] Oct 30 15:47:35.143349 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:47:35.143320 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod329ad3b4_3b1e_41e7_ae1b_12674d5c3715.slice/crio-717aa07e5457449e8034b9a4a191b6f4d3934fca3820f6fa375686ac8c3517e1 WatchSource:0}: Error finding container 717aa07e5457449e8034b9a4a191b6f4d3934fca3820f6fa375686ac8c3517e1: Status 404 returned error can't find the container with id 717aa07e5457449e8034b9a4a191b6f4d3934fca3820f6fa375686ac8c3517e1 Oct 30 15:47:35.159501 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:35.159476 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-metrics-certs\") pod \"maas-controller-5964675dcd-lswsg\" (UID: \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\") " pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 15:47:35.159613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:35.159505 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-webhook-cert\") pod \"maas-controller-5964675dcd-lswsg\" (UID: \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\") " pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 15:47:35.159613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:35.159540 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44zf9\" (UniqueName: \"kubernetes.io/projected/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-kube-api-access-44zf9\") pod \"maas-controller-5964675dcd-lswsg\" (UID: \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\") " pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 15:47:35.162035 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:35.162008 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-metrics-certs\") pod \"maas-controller-5964675dcd-lswsg\" (UID: \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\") " pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 15:47:35.162035 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:35.162022 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-webhook-cert\") pod \"maas-controller-5964675dcd-lswsg\" (UID: \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\") " pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 15:47:35.182925 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:35.182905 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44zf9\" (UniqueName: \"kubernetes.io/projected/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-kube-api-access-44zf9\") pod \"maas-controller-5964675dcd-lswsg\" (UID: \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\") " pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 15:47:35.252510 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:35.252491 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 15:47:35.470680 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:35.470648 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-qd7qg" event={"ID":"329ad3b4-3b1e-41e7-ae1b-12674d5c3715","Type":"ContainerStarted","Data":"717aa07e5457449e8034b9a4a191b6f4d3934fca3820f6fa375686ac8c3517e1"} Oct 30 15:47:35.549566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:35.549542 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5964675dcd-lswsg"] Oct 30 15:47:35.549990 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:47:35.549971 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5adc0947_063e_4c71_a2c6_d8395cc3b0d2.slice/crio-58a322e28366a061dbda8acb7eb9a15d00aae6259a1e22788a8590d5b5cd0a1a WatchSource:0}: Error finding container 58a322e28366a061dbda8acb7eb9a15d00aae6259a1e22788a8590d5b5cd0a1a: Status 404 returned error can't find the container with id 58a322e28366a061dbda8acb7eb9a15d00aae6259a1e22788a8590d5b5cd0a1a Oct 30 15:47:36.147104 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:36.147066 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:36.480292 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:36.480148 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5964675dcd-lswsg" event={"ID":"5adc0947-063e-4c71-a2c6-d8395cc3b0d2","Type":"ContainerStarted","Data":"58a322e28366a061dbda8acb7eb9a15d00aae6259a1e22788a8590d5b5cd0a1a"} Oct 30 15:47:38.366854 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:38.366765 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:38.492098 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:38.492059 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-qd7qg" event={"ID":"329ad3b4-3b1e-41e7-ae1b-12674d5c3715","Type":"ContainerStarted","Data":"af5be48f79e4292a7e64d041460228fa1d6234a5c6e734b39a4c04f46b15b713"} Oct 30 15:47:38.492311 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:38.492218 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:38.493566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:38.493539 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5964675dcd-lswsg" event={"ID":"5adc0947-063e-4c71-a2c6-d8395cc3b0d2","Type":"ContainerStarted","Data":"19110742045fe84ab2b11e12484af32385528c38e0148b5dcc58fe02a168da7f"} Oct 30 15:47:38.493707 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:38.493580 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 15:47:38.549736 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:38.549694 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-577f596567-qd7qg" podStartSLOduration=1.670307725 podStartE2EDuration="4.549681899s" totalImagesPullingTime="2.879374174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:47:38.549386192 +0000 UTC m=+574.700283223" watchObservedRunningTime="2026-10-30 15:47:38.549681899 +0000 UTC m=+574.700578930" Oct 30 15:47:38.555288 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:38.555265 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:38.595956 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:38.595919 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5964675dcd-lswsg" podStartSLOduration=1.954587047 podStartE2EDuration="4.59590801s" totalImagesPullingTime="2.641320963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:47:38.594286412 +0000 UTC m=+574.745183456" watchObservedRunningTime="2026-10-30 15:47:38.59590801 +0000 UTC m=+574.746805041" Oct 30 15:47:41.747555 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.747526 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw"] Oct 30 15:47:41.751284 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.751267 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:41.769259 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.769234 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw"] Oct 30 15:47:41.779706 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.779683 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\"" Oct 30 15:47:41.779975 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.779926 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\"" Oct 30 15:47:41.780112 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.779961 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\"" Oct 30 15:47:41.780254 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.779984 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-9p2dz\"" Oct 30 15:47:41.822842 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.822816 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-sdqhw\" (UID: \"2e87657e-b3bd-4a34-b354-3c0d0a740fc8\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:41.822991 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.822849 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-sdqhw\" (UID: \"2e87657e-b3bd-4a34-b354-3c0d0a740fc8\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:41.822991 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.822879 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-sdqhw\" (UID: \"2e87657e-b3bd-4a34-b354-3c0d0a740fc8\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:41.823074 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.823011 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbcql\" (UniqueName: \"kubernetes.io/projected/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-kube-api-access-qbcql\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-sdqhw\" (UID: \"2e87657e-b3bd-4a34-b354-3c0d0a740fc8\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:41.873352 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.873318 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:41.924478 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.924445 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbcql\" (UniqueName: \"kubernetes.io/projected/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-kube-api-access-qbcql\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-sdqhw\" (UID: \"2e87657e-b3bd-4a34-b354-3c0d0a740fc8\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:41.924633 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.924513 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-sdqhw\" (UID: \"2e87657e-b3bd-4a34-b354-3c0d0a740fc8\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:41.924633 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.924548 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-sdqhw\" (UID: \"2e87657e-b3bd-4a34-b354-3c0d0a740fc8\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:41.924633 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.924599 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-sdqhw\" (UID: \"2e87657e-b3bd-4a34-b354-3c0d0a740fc8\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:41.924633 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:47:41.924619 5013 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found Oct 30 15:47:41.924817 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:47:41.924686 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-tls-cert podName:2e87657e-b3bd-4a34-b354-3c0d0a740fc8 nodeName:}" failed. No retries permitted until 2026-10-30 15:47:42.424667672 +0000 UTC m=+578.575564699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-tls-cert") pod "usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" (UID: "2e87657e-b3bd-4a34-b354-3c0d0a740fc8") : secret "usage-logs-tenancy-proxy-tls" not found Oct 30 15:47:41.925348 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.925331 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-sdqhw\" (UID: \"2e87657e-b3bd-4a34-b354-3c0d0a740fc8\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:41.925521 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.925499 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-sdqhw\" (UID: \"2e87657e-b3bd-4a34-b354-3c0d0a740fc8\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:41.947854 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:41.947821 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbcql\" (UniqueName: \"kubernetes.io/projected/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-kube-api-access-qbcql\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-sdqhw\" (UID: \"2e87657e-b3bd-4a34-b354-3c0d0a740fc8\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:42.121980 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:42.121896 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:42.428884 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:42.428857 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-sdqhw\" (UID: \"2e87657e-b3bd-4a34-b354-3c0d0a740fc8\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:42.431523 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:42.431504 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2e87657e-b3bd-4a34-b354-3c0d0a740fc8-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-sdqhw\" (UID: \"2e87657e-b3bd-4a34-b354-3c0d0a740fc8\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:42.665845 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:42.665815 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:42.966988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:42.966941 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw"] Oct 30 15:47:42.968223 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:47:42.968182 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e87657e_b3bd_4a34_b354_3c0d0a740fc8.slice/crio-8a4f68a2fed3f9c7a07297ae09def7be0ec4fb05e411551efda14abaf15074c7 WatchSource:0}: Error finding container 8a4f68a2fed3f9c7a07297ae09def7be0ec4fb05e411551efda14abaf15074c7: Status 404 returned error can't find the container with id 8a4f68a2fed3f9c7a07297ae09def7be0ec4fb05e411551efda14abaf15074c7 Oct 30 15:47:43.518511 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:43.518479 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" event={"ID":"2e87657e-b3bd-4a34-b354-3c0d0a740fc8","Type":"ContainerStarted","Data":"8a4f68a2fed3f9c7a07297ae09def7be0ec4fb05e411551efda14abaf15074c7"} Oct 30 15:47:46.069164 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:46.069083 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:49.500319 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:49.500288 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 15:47:49.500777 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:49.500603 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:49.603401 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:49.603373 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-qd7qg"] Oct 30 15:47:49.603604 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:49.603581 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-577f596567-qd7qg" podUID="329ad3b4-3b1e-41e7-ae1b-12674d5c3715" containerName="manager" containerID="cri-o://af5be48f79e4292a7e64d041460228fa1d6234a5c6e734b39a4c04f46b15b713" gracePeriod=30 Oct 30 15:47:49.886519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:49.886488 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:50.005949 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.005910 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-kube-api-access-gnqkm\" (UniqueName: \"kubernetes.io/projected/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-kube-api-access-gnqkm\") pod \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\" (UID: \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\") " Oct 30 15:47:50.006134 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.006043 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-webhook-cert\" (UniqueName: \"kubernetes.io/secret/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-webhook-cert\") pod \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\" (UID: \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\") " Oct 30 15:47:50.006134 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.006082 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-metrics-certs\" (UniqueName: \"kubernetes.io/secret/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-metrics-certs\") pod \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\" (UID: \"329ad3b4-3b1e-41e7-ae1b-12674d5c3715\") " Oct 30 15:47:50.009743 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.009711 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-metrics-certs" pod "329ad3b4-3b1e-41e7-ae1b-12674d5c3715" (UID: "329ad3b4-3b1e-41e7-ae1b-12674d5c3715"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:47:50.010129 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.010101 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-kube-api-access-gnqkm" pod "329ad3b4-3b1e-41e7-ae1b-12674d5c3715" (UID: "329ad3b4-3b1e-41e7-ae1b-12674d5c3715"). InnerVolumeSpecName "kube-api-access-gnqkm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:47:50.011152 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.011113 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-webhook-cert" pod "329ad3b4-3b1e-41e7-ae1b-12674d5c3715" (UID: "329ad3b4-3b1e-41e7-ae1b-12674d5c3715"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:47:50.107303 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.107233 5013 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-webhook-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:47:50.107303 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.107260 5013 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-metrics-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:47:50.107303 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.107295 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnqkm\" (UniqueName: \"kubernetes.io/projected/329ad3b4-3b1e-41e7-ae1b-12674d5c3715-kube-api-access-gnqkm\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:47:50.560288 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.560248 5013 generic.go:356] "Generic (PLEG): container finished" podID="329ad3b4-3b1e-41e7-ae1b-12674d5c3715" containerID="af5be48f79e4292a7e64d041460228fa1d6234a5c6e734b39a4c04f46b15b713" exitCode=0 Oct 30 15:47:50.560731 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.560313 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-qd7qg" event={"ID":"329ad3b4-3b1e-41e7-ae1b-12674d5c3715","Type":"ContainerDied","Data":"af5be48f79e4292a7e64d041460228fa1d6234a5c6e734b39a4c04f46b15b713"} Oct 30 15:47:50.560731 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.560330 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-qd7qg" Oct 30 15:47:50.560731 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.560371 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-qd7qg" event={"ID":"329ad3b4-3b1e-41e7-ae1b-12674d5c3715","Type":"ContainerDied","Data":"717aa07e5457449e8034b9a4a191b6f4d3934fca3820f6fa375686ac8c3517e1"} Oct 30 15:47:50.560731 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.560400 5013 scope.go:122] "RemoveContainer" containerID="af5be48f79e4292a7e64d041460228fa1d6234a5c6e734b39a4c04f46b15b713" Oct 30 15:47:50.570390 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.570367 5013 scope.go:122] "RemoveContainer" containerID="af5be48f79e4292a7e64d041460228fa1d6234a5c6e734b39a4c04f46b15b713" Oct 30 15:47:50.570710 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:47:50.570678 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af5be48f79e4292a7e64d041460228fa1d6234a5c6e734b39a4c04f46b15b713\": container with ID starting with af5be48f79e4292a7e64d041460228fa1d6234a5c6e734b39a4c04f46b15b713 not found: ID does not exist" containerID="af5be48f79e4292a7e64d041460228fa1d6234a5c6e734b39a4c04f46b15b713" Oct 30 15:47:50.570801 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.570712 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af5be48f79e4292a7e64d041460228fa1d6234a5c6e734b39a4c04f46b15b713"} err="failed to get container status \"af5be48f79e4292a7e64d041460228fa1d6234a5c6e734b39a4c04f46b15b713\": rpc error: code = NotFound desc = could not find container \"af5be48f79e4292a7e64d041460228fa1d6234a5c6e734b39a4c04f46b15b713\": container with ID starting with af5be48f79e4292a7e64d041460228fa1d6234a5c6e734b39a4c04f46b15b713 not found: ID does not exist" Oct 30 15:47:50.614576 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.614550 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-qd7qg"] Oct 30 15:47:50.626633 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:50.626611 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-577f596567-qd7qg"] Oct 30 15:47:52.005027 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:52.004994 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="329ad3b4-3b1e-41e7-ae1b-12674d5c3715" path="/var/lib/kubelet/pods/329ad3b4-3b1e-41e7-ae1b-12674d5c3715/volumes" Oct 30 15:47:54.311326 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:54.311281 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:47:54.584189 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:54.584098 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" event={"ID":"2e87657e-b3bd-4a34-b354-3c0d0a740fc8","Type":"ContainerStarted","Data":"bdedfa55d5f13d324f5720ff3ac95ac44e636dd4a331a449499c7728dcdaa1f9"} Oct 30 15:47:54.584382 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:54.584236 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:47:54.625015 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:47:54.624975 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" podStartSLOduration=2.782162977 podStartE2EDuration="13.624962543s" totalImagesPullingTime="10.842799566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:47:54.624462081 +0000 UTC m=+590.775359111" watchObservedRunningTime="2026-10-30 15:47:54.624962543 +0000 UTC m=+590.775859574" Oct 30 15:47:56.975714 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:47:56.975663 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:48:04.042058 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:04.042027 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 15:48:04.044605 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:04.044574 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 15:48:05.520151 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:05.520115 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:48:05.593740 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:05.593719 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-sdqhw" Oct 30 15:48:12.946432 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:12.946399 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:48:13.195949 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:13.195915 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:48:14.345595 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.345562 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:48:14.529809 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.529780 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:48:14.828889 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.828856 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-258kv"] Oct 30 15:48:14.829299 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.829286 5013 cpu_manager.go:414] "containerMap: removing container" podUID="329ad3b4-3b1e-41e7-ae1b-12674d5c3715" containerName="manager" Oct 30 15:48:14.829366 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.829300 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="329ad3b4-3b1e-41e7-ae1b-12674d5c3715" containerName="manager" Oct 30 15:48:14.829366 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.829365 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="329ad3b4-3b1e-41e7-ae1b-12674d5c3715" containerName="manager" Oct 30 15:48:14.835977 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.835959 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:14.853566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.853545 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-258kv"] Oct 30 15:48:14.866482 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.866458 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gp5lt\"" Oct 30 15:48:14.866482 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.866486 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Oct 30 15:48:14.866671 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.866592 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Oct 30 15:48:14.893112 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.893089 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-metrics-certs\") pod \"maas-api-5c48877c75-258kv\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:14.893248 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.893130 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-openshift-service-ca\") pod \"maas-api-5c48877c75-258kv\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:14.893248 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.893167 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfsm5\" (UniqueName: \"kubernetes.io/projected/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-kube-api-access-qfsm5\") pod \"maas-api-5c48877c75-258kv\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:14.893328 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.893308 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-maas-api-tls\") pod \"maas-api-5c48877c75-258kv\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:14.993683 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.993654 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-metrics-certs\") pod \"maas-api-5c48877c75-258kv\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:14.993826 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.993691 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-openshift-service-ca\") pod \"maas-api-5c48877c75-258kv\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:14.993826 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.993717 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfsm5\" (UniqueName: \"kubernetes.io/projected/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-kube-api-access-qfsm5\") pod \"maas-api-5c48877c75-258kv\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:14.993826 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.993800 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-maas-api-tls\") pod \"maas-api-5c48877c75-258kv\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:14.994321 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.994289 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-openshift-service-ca\") pod \"maas-api-5c48877c75-258kv\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:14.996467 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.996443 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-metrics-certs\") pod \"maas-api-5c48877c75-258kv\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:14.996467 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:14.996456 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-maas-api-tls\") pod \"maas-api-5c48877c75-258kv\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:15.016945 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:15.016919 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfsm5\" (UniqueName: \"kubernetes.io/projected/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-kube-api-access-qfsm5\") pod \"maas-api-5c48877c75-258kv\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:15.148367 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:15.148329 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:15.452406 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:15.452378 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-258kv"] Oct 30 15:48:15.453296 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:48:15.453270 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d62df41_e0d5_4376_912f_f61f8d1fbd2d.slice/crio-0dc8d418f5119a5b16f6f6054f9379a1a3573c16d8dc284b03db8db05a0bd90d WatchSource:0}: Error finding container 0dc8d418f5119a5b16f6f6054f9379a1a3573c16d8dc284b03db8db05a0bd90d: Status 404 returned error can't find the container with id 0dc8d418f5119a5b16f6f6054f9379a1a3573c16d8dc284b03db8db05a0bd90d Oct 30 15:48:15.454677 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:15.454660 5013 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 15:48:15.678493 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:15.678459 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" event={"ID":"8d62df41-e0d5-4376-912f-f61f8d1fbd2d","Type":"ContainerStarted","Data":"0dc8d418f5119a5b16f6f6054f9379a1a3573c16d8dc284b03db8db05a0bd90d"} Oct 30 15:48:17.690300 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:17.690267 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" event={"ID":"8d62df41-e0d5-4376-912f-f61f8d1fbd2d","Type":"ContainerStarted","Data":"b3de0a1c1e760905faa28a836fd2efa3b0611a33069b8189e30d6a53945bd793"} Oct 30 15:48:17.690752 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:17.690463 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:17.742081 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:17.742034 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" podStartSLOduration=2.209425152 podStartE2EDuration="3.742020684s" totalImagesPullingTime="1.532595532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:48:17.741496572 +0000 UTC m=+613.892393602" watchObservedRunningTime="2026-10-30 15:48:17.742020684 +0000 UTC m=+613.892917714" Oct 30 15:48:23.701781 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:23.701751 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 15:48:24.976185 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:24.976153 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-vwt89"] Oct 30 15:48:25.114635 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:25.114601 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-vwt89"] Oct 30 15:48:25.114809 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:25.114717 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-vwt89" Oct 30 15:48:25.184108 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:25.184081 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlm24\" (UniqueName: \"kubernetes.io/projected/fcb0b34e-a57f-4375-b244-ecdc3f1848c4-kube-api-access-xlm24\") pod \"llmisvc-controller-manager-6466c554d8-vwt89\" (UID: \"fcb0b34e-a57f-4375-b244-ecdc3f1848c4\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-vwt89" Oct 30 15:48:25.184290 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:25.184115 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fcb0b34e-a57f-4375-b244-ecdc3f1848c4-cert\") pod \"llmisvc-controller-manager-6466c554d8-vwt89\" (UID: \"fcb0b34e-a57f-4375-b244-ecdc3f1848c4\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-vwt89" Oct 30 15:48:25.284801 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:25.284733 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlm24\" (UniqueName: \"kubernetes.io/projected/fcb0b34e-a57f-4375-b244-ecdc3f1848c4-kube-api-access-xlm24\") pod \"llmisvc-controller-manager-6466c554d8-vwt89\" (UID: \"fcb0b34e-a57f-4375-b244-ecdc3f1848c4\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-vwt89" Oct 30 15:48:25.284801 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:25.284766 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fcb0b34e-a57f-4375-b244-ecdc3f1848c4-cert\") pod \"llmisvc-controller-manager-6466c554d8-vwt89\" (UID: \"fcb0b34e-a57f-4375-b244-ecdc3f1848c4\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-vwt89" Oct 30 15:48:25.287220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:25.287184 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fcb0b34e-a57f-4375-b244-ecdc3f1848c4-cert\") pod \"llmisvc-controller-manager-6466c554d8-vwt89\" (UID: \"fcb0b34e-a57f-4375-b244-ecdc3f1848c4\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-vwt89" Oct 30 15:48:25.307862 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:25.307835 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlm24\" (UniqueName: \"kubernetes.io/projected/fcb0b34e-a57f-4375-b244-ecdc3f1848c4-kube-api-access-xlm24\") pod \"llmisvc-controller-manager-6466c554d8-vwt89\" (UID: \"fcb0b34e-a57f-4375-b244-ecdc3f1848c4\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-vwt89" Oct 30 15:48:25.461593 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:25.461568 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-vwt89" Oct 30 15:48:25.812127 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:25.812095 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-vwt89"] Oct 30 15:48:25.812598 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:48:25.812567 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcb0b34e_a57f_4375_b244_ecdc3f1848c4.slice/crio-3e88cd76a93ee53f815430f0f67ebcc56e19801e68ab3caf4d6af3918b4e3b00 WatchSource:0}: Error finding container 3e88cd76a93ee53f815430f0f67ebcc56e19801e68ab3caf4d6af3918b4e3b00: Status 404 returned error can't find the container with id 3e88cd76a93ee53f815430f0f67ebcc56e19801e68ab3caf4d6af3918b4e3b00 Oct 30 15:48:26.732979 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:26.732945 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-vwt89" event={"ID":"fcb0b34e-a57f-4375-b244-ecdc3f1848c4","Type":"ContainerStarted","Data":"e2d7b16735ea46e58049ab17c9ebbee0f47b3c6bf66f06228990acfd29a4aefc"} Oct 30 15:48:26.733464 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:26.733005 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-vwt89" event={"ID":"fcb0b34e-a57f-4375-b244-ecdc3f1848c4","Type":"ContainerStarted","Data":"3e88cd76a93ee53f815430f0f67ebcc56e19801e68ab3caf4d6af3918b4e3b00"} Oct 30 15:48:26.733464 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:26.733046 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-vwt89" Oct 30 15:48:26.773775 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:26.773721 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-vwt89" podStartSLOduration=2.28969125 podStartE2EDuration="2.773709078s" totalImagesPullingTime="484.017828ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:48:26.773047098 +0000 UTC m=+622.923944129" watchObservedRunningTime="2026-10-30 15:48:26.773709078 +0000 UTC m=+622.924606109" Oct 30 15:48:26.976125 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:48:26.976076 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:48:54.440146 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:54.440117 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:48:56.976623 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:48:56.976587 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:48:57.740481 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:57.740455 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-vwt89" Oct 30 15:48:57.809216 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:57.809173 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh"] Oct 30 15:48:57.809415 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:57.809394 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" podUID="fdb844e1-85f8-4d39-97b1-bfd946ef4d34" containerName="manager" containerID="cri-o://9852920792e00152c7e5437e48a1d55a919c2f86a263bb45ef0156dcc971c6cf" gracePeriod=30 Oct 30 15:48:58.054032 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.054009 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" Oct 30 15:48:58.158022 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.157995 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fdb844e1-85f8-4d39-97b1-bfd946ef4d34-cert\" (UniqueName: \"kubernetes.io/secret/fdb844e1-85f8-4d39-97b1-bfd946ef4d34-cert\") pod \"fdb844e1-85f8-4d39-97b1-bfd946ef4d34\" (UID: \"fdb844e1-85f8-4d39-97b1-bfd946ef4d34\") " Oct 30 15:48:58.158154 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.158034 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdb844e1-85f8-4d39-97b1-bfd946ef4d34-kube-api-access-6sf22\" (UniqueName: \"kubernetes.io/projected/fdb844e1-85f8-4d39-97b1-bfd946ef4d34-kube-api-access-6sf22\") pod \"fdb844e1-85f8-4d39-97b1-bfd946ef4d34\" (UID: \"fdb844e1-85f8-4d39-97b1-bfd946ef4d34\") " Oct 30 15:48:58.160165 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.160139 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdb844e1-85f8-4d39-97b1-bfd946ef4d34-cert" pod "fdb844e1-85f8-4d39-97b1-bfd946ef4d34" (UID: "fdb844e1-85f8-4d39-97b1-bfd946ef4d34"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:48:58.160300 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.160273 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdb844e1-85f8-4d39-97b1-bfd946ef4d34-kube-api-access-6sf22" pod "fdb844e1-85f8-4d39-97b1-bfd946ef4d34" (UID: "fdb844e1-85f8-4d39-97b1-bfd946ef4d34"). InnerVolumeSpecName "kube-api-access-6sf22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:48:58.259320 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.259301 5013 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fdb844e1-85f8-4d39-97b1-bfd946ef4d34-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:48:58.259320 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.259320 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sf22\" (UniqueName: \"kubernetes.io/projected/fdb844e1-85f8-4d39-97b1-bfd946ef4d34-kube-api-access-6sf22\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:48:58.709168 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.709138 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:48:58.875542 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.875496 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:48:58.877806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.877777 5013 generic.go:356] "Generic (PLEG): container finished" podID="fdb844e1-85f8-4d39-97b1-bfd946ef4d34" containerID="9852920792e00152c7e5437e48a1d55a919c2f86a263bb45ef0156dcc971c6cf" exitCode=0 Oct 30 15:48:58.877933 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.877827 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" Oct 30 15:48:58.877933 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.877866 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" event={"ID":"fdb844e1-85f8-4d39-97b1-bfd946ef4d34","Type":"ContainerDied","Data":"9852920792e00152c7e5437e48a1d55a919c2f86a263bb45ef0156dcc971c6cf"} Oct 30 15:48:58.877933 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.877917 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh" event={"ID":"fdb844e1-85f8-4d39-97b1-bfd946ef4d34","Type":"ContainerDied","Data":"43b03fa9886c427be5933f767082aeecc58f381162f26c64525d1811b9cea88c"} Oct 30 15:48:58.878055 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.877945 5013 scope.go:122] "RemoveContainer" containerID="9852920792e00152c7e5437e48a1d55a919c2f86a263bb45ef0156dcc971c6cf" Oct 30 15:48:58.886909 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.886892 5013 scope.go:122] "RemoveContainer" containerID="9852920792e00152c7e5437e48a1d55a919c2f86a263bb45ef0156dcc971c6cf" Oct 30 15:48:58.887151 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:48:58.887135 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9852920792e00152c7e5437e48a1d55a919c2f86a263bb45ef0156dcc971c6cf\": container with ID starting with 9852920792e00152c7e5437e48a1d55a919c2f86a263bb45ef0156dcc971c6cf not found: ID does not exist" containerID="9852920792e00152c7e5437e48a1d55a919c2f86a263bb45ef0156dcc971c6cf" Oct 30 15:48:58.887229 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.887160 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9852920792e00152c7e5437e48a1d55a919c2f86a263bb45ef0156dcc971c6cf"} err="failed to get container status \"9852920792e00152c7e5437e48a1d55a919c2f86a263bb45ef0156dcc971c6cf\": rpc error: code = NotFound desc = could not find container \"9852920792e00152c7e5437e48a1d55a919c2f86a263bb45ef0156dcc971c6cf\": container with ID starting with 9852920792e00152c7e5437e48a1d55a919c2f86a263bb45ef0156dcc971c6cf not found: ID does not exist" Oct 30 15:48:58.926421 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.926400 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh"] Oct 30 15:48:58.938094 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:48:58.938069 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-9g6sh"] Oct 30 15:49:00.003929 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:00.003897 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fdb844e1-85f8-4d39-97b1-bfd946ef4d34" path="/var/lib/kubelet/pods/fdb844e1-85f8-4d39-97b1-bfd946ef4d34/volumes" Oct 30 15:49:17.608612 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:17.608516 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:49:19.412584 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:19.412535 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:49:21.189413 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:21.189369 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:49:22.689485 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:22.689450 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:49:24.720703 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:24.720667 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:49:26.829864 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:26.829831 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:49:26.978391 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:49:26.978360 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:49:28.326721 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:28.326693 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:49:45.052079 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:45.052047 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:49:45.232753 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:45.232719 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:49:50.129516 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:50.129468 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:49:50.298949 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:50.298920 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:49:51.990389 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:51.990360 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz"] Oct 30 15:49:51.990756 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:51.990710 5013 cpu_manager.go:414] "containerMap: removing container" podUID="fdb844e1-85f8-4d39-97b1-bfd946ef4d34" containerName="manager" Oct 30 15:49:51.990756 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:51.990720 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdb844e1-85f8-4d39-97b1-bfd946ef4d34" containerName="manager" Oct 30 15:49:51.990845 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:51.990810 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdb844e1-85f8-4d39-97b1-bfd946ef4d34" containerName="manager" Oct 30 15:49:51.993882 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:51.993863 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.004836 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.004815 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.004951 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.004849 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.004951 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.004875 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.004951 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.004923 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f333930d-c55e-4cff-94b4-85861e2521b8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.004951 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.004944 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f333930d-c55e-4cff-94b4-85861e2521b8-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.005145 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.004987 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f333930d-c55e-4cff-94b4-85861e2521b8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.005234 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.005173 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.005303 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.005255 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.005358 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.005298 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sb64\" (UniqueName: \"kubernetes.io/projected/f333930d-c55e-4cff-94b4-85861e2521b8-kube-api-access-4sb64\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.022407 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.022385 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz"] Oct 30 15:49:52.097869 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.097844 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Oct 30 15:49:52.097990 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.097911 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-x4s56\"" Oct 30 15:49:52.106524 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.106501 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.106595 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.106534 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sb64\" (UniqueName: \"kubernetes.io/projected/f333930d-c55e-4cff-94b4-85861e2521b8-kube-api-access-4sb64\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.106595 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.106563 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.106595 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.106588 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.106743 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.106611 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.106743 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.106654 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f333930d-c55e-4cff-94b4-85861e2521b8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.106743 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.106675 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f333930d-c55e-4cff-94b4-85861e2521b8-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.106743 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.106715 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f333930d-c55e-4cff-94b4-85861e2521b8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.106926 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.106761 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.106983 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.106960 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.107039 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.107001 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.107093 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.107056 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.107244 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.107189 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.107459 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.107443 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f333930d-c55e-4cff-94b4-85861e2521b8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.109057 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.109041 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f333930d-c55e-4cff-94b4-85861e2521b8-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.109330 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.109311 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f333930d-c55e-4cff-94b4-85861e2521b8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.129602 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.129581 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sb64\" (UniqueName: \"kubernetes.io/projected/f333930d-c55e-4cff-94b4-85861e2521b8-kube-api-access-4sb64\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.130442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.130412 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f333930d-c55e-4cff-94b4-85861e2521b8-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fljdz\" (UID: \"f333930d-c55e-4cff-94b4-85861e2521b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.307525 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.307442 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:52.621894 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:52.621871 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz"] Oct 30 15:49:52.623065 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:49:52.623044 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf333930d_c55e_4cff_94b4_85861e2521b8.slice/crio-6a365cff572c77a4fa215025bd04bb757f2f94418b3384ae791d026e8fbbc01d WatchSource:0}: Error finding container 6a365cff572c77a4fa215025bd04bb757f2f94418b3384ae791d026e8fbbc01d: Status 404 returned error can't find the container with id 6a365cff572c77a4fa215025bd04bb757f2f94418b3384ae791d026e8fbbc01d Oct 30 15:49:53.132868 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:53.132838 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" event={"ID":"f333930d-c55e-4cff-94b4-85861e2521b8","Type":"ContainerStarted","Data":"6a365cff572c77a4fa215025bd04bb757f2f94418b3384ae791d026e8fbbc01d"} Oct 30 15:49:54.570561 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:54.570518 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:49:55.400478 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:55.400446 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:49:55.400583 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:55.400513 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:49:55.400583 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:55.400550 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:49:56.149222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:56.149172 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" event={"ID":"f333930d-c55e-4cff-94b4-85861e2521b8","Type":"ContainerStarted","Data":"b883aba493871911edd7ca29ea24695c0a8b298d1c35834611c438e25b49667a"} Oct 30 15:49:56.220347 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:56.220301 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" podStartSLOduration=2.445666856 podStartE2EDuration="5.220286809s" totalImagesPullingTime="2.774619953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:49:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:49:56.207833399 +0000 UTC m=+712.358730440" watchObservedRunningTime="2026-10-30 15:49:56.220286809 +0000 UTC m=+712.371183840" Oct 30 15:49:56.308205 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:56.308163 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:49:56.309684 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:56.309661 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-fljdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Oct 30 15:49:56.309793 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:56.309704 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" podUID="f333930d-c55e-4cff-94b4-85861e2521b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Oct 30 15:49:56.998766 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:49:56.998725 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:49:57.308402 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:57.308320 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-fljdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Oct 30 15:49:57.308402 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:57.308386 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" podUID="f333930d-c55e-4cff-94b4-85861e2521b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Oct 30 15:49:58.308253 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:58.308220 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-fljdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Oct 30 15:49:58.308513 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:58.308279 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" podUID="f333930d-c55e-4cff-94b4-85861e2521b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Oct 30 15:49:59.307828 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:59.307790 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-fljdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Oct 30 15:49:59.308003 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:49:59.307856 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" podUID="f333930d-c55e-4cff-94b4-85861e2521b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Oct 30 15:50:00.308519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:00.308488 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-fljdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Oct 30 15:50:00.308951 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:00.308557 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" podUID="f333930d-c55e-4cff-94b4-85861e2521b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Oct 30 15:50:01.308170 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:01.308136 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-fljdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Oct 30 15:50:01.308373 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:01.308230 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" podUID="f333930d-c55e-4cff-94b4-85861e2521b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Oct 30 15:50:02.307827 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:02.307794 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-fljdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Oct 30 15:50:02.308215 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:02.307844 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" podUID="f333930d-c55e-4cff-94b4-85861e2521b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Oct 30 15:50:03.636489 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:03.636459 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:50:03.636973 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:03.636863 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:50:03.637829 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:03.637809 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fljdz" Oct 30 15:50:07.036457 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.036420 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd"] Oct 30 15:50:07.040585 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.040545 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.064772 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.064748 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd"] Oct 30 15:50:07.161034 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.160988 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.161239 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.161046 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.161239 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.161103 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.161363 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.161245 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8rvb\" (UniqueName: \"kubernetes.io/projected/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-kube-api-access-v8rvb\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.161363 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.161283 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.161446 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.161375 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.161446 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.161393 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.161446 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.161428 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.161591 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.161513 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.262422 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.262383 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.262678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.262451 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.262678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.262479 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.262678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.262511 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.262678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.262536 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.262678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.262595 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8rvb\" (UniqueName: \"kubernetes.io/projected/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-kube-api-access-v8rvb\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.262678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.262624 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.262678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.262664 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.263085 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.262690 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.263085 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.262787 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.263085 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.262934 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.263085 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.262994 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.263290 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.263087 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.263573 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.263556 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.265003 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.264979 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.265597 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.265576 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.285943 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.285915 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.286703 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.286645 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8rvb\" (UniqueName: \"kubernetes.io/projected/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-kube-api-access-v8rvb\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-5s8sd\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.354685 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.354658 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:07.669283 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.669257 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd"] Oct 30 15:50:07.670280 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:50:07.670254 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a27f82_3d2b_4bc3_9e2c_3fd0f54da2f9.slice/crio-bf2be9765537f2527648dc85a72c788431086034892b7c823063a99d4c4346c7 WatchSource:0}: Error finding container bf2be9765537f2527648dc85a72c788431086034892b7c823063a99d4c4346c7: Status 404 returned error can't find the container with id bf2be9765537f2527648dc85a72c788431086034892b7c823063a99d4c4346c7 Oct 30 15:50:07.672774 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.672740 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:50:07.672854 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.672807 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:50:07.672854 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:07.672835 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:50:08.220469 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:08.220431 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" event={"ID":"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9","Type":"ContainerStarted","Data":"7e524e5979b2a11390fa39beb6ec1ed10a3e849055fa666d51e5fa5cbb4f3081"} Oct 30 15:50:08.220979 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:08.220499 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" event={"ID":"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9","Type":"ContainerStarted","Data":"bf2be9765537f2527648dc85a72c788431086034892b7c823063a99d4c4346c7"} Oct 30 15:50:08.300795 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:08.300739 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" podStartSLOduration=1.300724523 podStartE2EDuration="1.300724523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:08.288475698 +0000 UTC m=+724.439372730" watchObservedRunningTime="2026-10-30 15:50:08.300724523 +0000 UTC m=+724.451621611" Oct 30 15:50:08.355074 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:08.355049 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:08.356732 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:08.356707 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Oct 30 15:50:08.356832 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:08.356765 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" podUID="04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Oct 30 15:50:09.355837 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:09.355805 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Oct 30 15:50:09.356241 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:09.355855 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" podUID="04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Oct 30 15:50:10.355212 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:10.355164 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Oct 30 15:50:10.355385 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:10.355235 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" podUID="04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Oct 30 15:50:11.355039 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:11.355000 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Oct 30 15:50:11.355475 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:11.355047 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" podUID="04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Oct 30 15:50:12.355630 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:12.355598 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Oct 30 15:50:12.355988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:12.355646 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" podUID="04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Oct 30 15:50:13.355003 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:13.354972 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Oct 30 15:50:13.355179 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:13.355027 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" podUID="04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Oct 30 15:50:14.355958 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:14.355913 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Oct 30 15:50:14.356485 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:14.355964 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" podUID="04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Oct 30 15:50:14.660043 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:14.660003 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:15.192548 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:15.192499 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:15.845119 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:15.845087 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:15.845661 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:15.845641 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:15.846438 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:15.846421 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:50:16.865384 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:16.865352 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:17.032435 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:17.032406 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:18.558486 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:18.558452 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:18.733846 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:18.733812 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:26.977002 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:50:26.976960 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:50:39.912884 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:39.912849 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:40.076219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:40.076175 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:40.589013 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:40.588984 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:40.857813 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:40.857738 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:41.064168 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:41.064140 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:41.274532 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:41.274455 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:41.488214 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:41.488176 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:41.654352 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:41.654314 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:44.270049 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:44.269984 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:44.470218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:44.470162 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:44.753864 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:44.753836 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:44.930828 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:44.930791 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:45.112729 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:45.112654 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:45.296082 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:45.296042 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:45.602467 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:45.602435 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:45.790738 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:45.790705 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:45.965422 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:45.965393 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:46.449716 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:46.449686 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:46.645858 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:46.645831 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:47.924676 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:47.924644 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb"] Oct 30 15:50:47.928403 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:47.928383 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:47.934964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:47.934942 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:47.942958 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:47.942939 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-040f9a-serving-cert\"" Oct 30 15:50:47.957014 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:47.956990 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb"] Oct 30 15:50:48.037641 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.037613 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4facca78-e175-41fe-8e57-5a70e801a751-maas-api-tls\") pod \"maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:48.037807 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.037651 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4facca78-e175-41fe-8e57-5a70e801a751-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:48.037807 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.037758 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn6rv\" (UniqueName: \"kubernetes.io/projected/4facca78-e175-41fe-8e57-5a70e801a751-kube-api-access-cn6rv\") pod \"maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:48.037807 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.037806 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4facca78-e175-41fe-8e57-5a70e801a751-metrics-certs\") pod \"maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:48.101431 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.101385 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:48.139036 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.139004 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn6rv\" (UniqueName: \"kubernetes.io/projected/4facca78-e175-41fe-8e57-5a70e801a751-kube-api-access-cn6rv\") pod \"maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:48.139410 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.139366 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4facca78-e175-41fe-8e57-5a70e801a751-metrics-certs\") pod \"maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:48.140374 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.140353 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4facca78-e175-41fe-8e57-5a70e801a751-maas-api-tls\") pod \"maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:48.140571 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.140545 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4facca78-e175-41fe-8e57-5a70e801a751-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:48.141897 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.141876 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4facca78-e175-41fe-8e57-5a70e801a751-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:48.143942 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.143921 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4facca78-e175-41fe-8e57-5a70e801a751-maas-api-tls\") pod \"maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:48.144608 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.144585 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4facca78-e175-41fe-8e57-5a70e801a751-metrics-certs\") pod \"maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:48.164141 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.164115 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn6rv\" (UniqueName: \"kubernetes.io/projected/4facca78-e175-41fe-8e57-5a70e801a751-kube-api-access-cn6rv\") pod \"maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:48.242138 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.242067 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:48.260651 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.260630 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:48.540504 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.540476 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb"] Oct 30 15:50:48.541814 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:50:48.541792 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4facca78_e175_41fe_8e57_5a70e801a751.slice/crio-3ca3ca561c9b378799b22b04b8dce8b346811f178fb35b9ab185cfb9c379a3c8 WatchSource:0}: Error finding container 3ca3ca561c9b378799b22b04b8dce8b346811f178fb35b9ab185cfb9c379a3c8: Status 404 returned error can't find the container with id 3ca3ca561c9b378799b22b04b8dce8b346811f178fb35b9ab185cfb9c379a3c8 Oct 30 15:50:48.810111 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:48.810025 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:49.001156 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:49.001124 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:49.423847 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:49.423816 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" event={"ID":"4facca78-e175-41fe-8e57-5a70e801a751","Type":"ContainerStarted","Data":"1201f5e081673511312bc027288403837fd277724f9afba939759fd09faa5ac6"} Oct 30 15:50:49.423993 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:49.423874 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" event={"ID":"4facca78-e175-41fe-8e57-5a70e801a751","Type":"ContainerStarted","Data":"3ca3ca561c9b378799b22b04b8dce8b346811f178fb35b9ab185cfb9c379a3c8"} Oct 30 15:50:49.423993 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:49.423899 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:49.465504 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:49.465458 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" podStartSLOduration=2.066052874 podStartE2EDuration="2.465443577s" totalImagesPullingTime="399.390703ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:49.464754733 +0000 UTC m=+765.615651764" watchObservedRunningTime="2026-10-30 15:50:49.465443577 +0000 UTC m=+765.616340607" Oct 30 15:50:49.943087 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:49.943058 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch"] Oct 30 15:50:49.946636 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:49.946617 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:49.961584 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:49.961568 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-9e2b29-serving-cert\"" Oct 30 15:50:49.975296 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:49.975267 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch"] Oct 30 15:50:50.061566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.061537 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtdc2\" (UniqueName: \"kubernetes.io/projected/2a71f35c-8390-40b1-b1d3-828d7330eed9-kube-api-access-xtdc2\") pod \"maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:50.061914 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.061578 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a71f35c-8390-40b1-b1d3-828d7330eed9-maas-api-tls\") pod \"maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:50.061914 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.061596 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a71f35c-8390-40b1-b1d3-828d7330eed9-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:50.061914 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.061697 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a71f35c-8390-40b1-b1d3-828d7330eed9-metrics-certs\") pod \"maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:50.106881 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.106855 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:50.162969 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.162938 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a71f35c-8390-40b1-b1d3-828d7330eed9-metrics-certs\") pod \"maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:50.163152 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.163018 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtdc2\" (UniqueName: \"kubernetes.io/projected/2a71f35c-8390-40b1-b1d3-828d7330eed9-kube-api-access-xtdc2\") pod \"maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:50.163152 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.163058 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a71f35c-8390-40b1-b1d3-828d7330eed9-maas-api-tls\") pod \"maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:50.163152 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.163076 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a71f35c-8390-40b1-b1d3-828d7330eed9-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:50.163655 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.163635 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a71f35c-8390-40b1-b1d3-828d7330eed9-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:50.166290 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.166266 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a71f35c-8390-40b1-b1d3-828d7330eed9-metrics-certs\") pod \"maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:50.166415 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.166293 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a71f35c-8390-40b1-b1d3-828d7330eed9-maas-api-tls\") pod \"maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:50.186819 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.186795 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtdc2\" (UniqueName: \"kubernetes.io/projected/2a71f35c-8390-40b1-b1d3-828d7330eed9-kube-api-access-xtdc2\") pod \"maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:50.260053 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.259989 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:50.269404 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.269382 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:50.443688 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.443650 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:50.579451 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:50.579427 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch"] Oct 30 15:50:50.581053 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:50:50.581027 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a71f35c_8390_40b1_b1d3_828d7330eed9.slice/crio-878220ebd6844ab4e1f9fd5f979c3078bf840abd0786a40f694c4f2498dde33d WatchSource:0}: Error finding container 878220ebd6844ab4e1f9fd5f979c3078bf840abd0786a40f694c4f2498dde33d: Status 404 returned error can't find the container with id 878220ebd6844ab4e1f9fd5f979c3078bf840abd0786a40f694c4f2498dde33d Oct 30 15:50:51.156589 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:51.156507 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:51.333395 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:51.333354 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:51.435656 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:51.435570 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" event={"ID":"2a71f35c-8390-40b1-b1d3-828d7330eed9","Type":"ContainerStarted","Data":"0e003aa8b03e1ed66a45db53d563a096c161a039f4fd8e3d229912a384f63087"} Oct 30 15:50:51.435656 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:51.435634 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" event={"ID":"2a71f35c-8390-40b1-b1d3-828d7330eed9","Type":"ContainerStarted","Data":"878220ebd6844ab4e1f9fd5f979c3078bf840abd0786a40f694c4f2498dde33d"} Oct 30 15:50:51.435875 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:51.435764 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:51.478729 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:51.478686 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" podStartSLOduration=2.163065614 podStartE2EDuration="2.478673323s" totalImagesPullingTime="315.607709ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:51.476890561 +0000 UTC m=+767.627787588" watchObservedRunningTime="2026-10-30 15:50:51.478673323 +0000 UTC m=+767.629570354" Oct 30 15:50:51.861421 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:51.861345 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j"] Oct 30 15:50:51.865029 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:51.865011 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:51.881439 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:51.881416 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-ba9db9-serving-cert\"" Oct 30 15:50:51.884722 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:51.884695 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j"] Oct 30 15:50:51.979875 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:51.979837 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-metrics-certs\") pod \"maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:51.980028 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:51.979958 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-maas-api-tls\") pod \"maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:51.980028 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:51.979979 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:51.980103 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:51.980071 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrcmk\" (UniqueName: \"kubernetes.io/projected/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-kube-api-access-jrcmk\") pod \"maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:52.014742 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:52.014716 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:52.081310 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:52.081284 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrcmk\" (UniqueName: \"kubernetes.io/projected/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-kube-api-access-jrcmk\") pod \"maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:52.081435 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:52.081327 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-metrics-certs\") pod \"maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:52.081435 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:52.081378 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-maas-api-tls\") pod \"maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:52.081435 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:52.081392 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:52.082017 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:52.081986 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:52.083919 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:52.083900 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-maas-api-tls\") pod \"maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:52.084009 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:52.083990 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-metrics-certs\") pod \"maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:52.103967 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:52.103946 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrcmk\" (UniqueName: \"kubernetes.io/projected/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-kube-api-access-jrcmk\") pod \"maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:52.178506 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:52.178479 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:52.180532 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:52.180508 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:52.347560 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:52.347512 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:52.483874 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:52.483845 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j"] Oct 30 15:50:52.485468 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:50:52.485441 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f037ba0_aee4_4f9e_8b48_a3e7c1520671.slice/crio-b5b1d2a49ef7b6f9925f270d886bf8f66ca51f221577715491d516954c36700b WatchSource:0}: Error finding container b5b1d2a49ef7b6f9925f270d886bf8f66ca51f221577715491d516954c36700b: Status 404 returned error can't find the container with id b5b1d2a49ef7b6f9925f270d886bf8f66ca51f221577715491d516954c36700b Oct 30 15:50:53.225907 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:53.225878 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:53.395878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:53.395851 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:53.447807 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:53.447775 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" event={"ID":"1f037ba0-aee4-4f9e-8b48-a3e7c1520671","Type":"ContainerStarted","Data":"242c362c573a6076f8b7b85902f6caa1b9db909a7c6f142c3d26e6246b8ce4ec"} Oct 30 15:50:53.447983 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:53.447828 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:50:53.447983 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:53.447850 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" event={"ID":"1f037ba0-aee4-4f9e-8b48-a3e7c1520671","Type":"ContainerStarted","Data":"b5b1d2a49ef7b6f9925f270d886bf8f66ca51f221577715491d516954c36700b"} Oct 30 15:50:53.490934 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:53.490820 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" podStartSLOduration=2.20764387 podStartE2EDuration="2.490805848s" totalImagesPullingTime="283.161978ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:53.490300703 +0000 UTC m=+769.641197739" watchObservedRunningTime="2026-10-30 15:50:53.490805848 +0000 UTC m=+769.641702878" Oct 30 15:50:53.813426 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:53.813350 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:54.151240 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:54.151206 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:54.318759 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:54.318728 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:54.693027 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:54.692992 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:55.435805 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:55.435775 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:50:55.523072 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:55.523020 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:55.682516 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:55.682441 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:56.977978 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:50:56.977935 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:50:57.390627 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.390596 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6"] Oct 30 15:50:57.394492 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.394471 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:57.412593 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.412561 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-a2b3f1-serving-cert\"" Oct 30 15:50:57.424149 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.424125 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6"] Oct 30 15:50:57.446839 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.446819 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:50:57.533365 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.533330 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhw2z\" (UniqueName: \"kubernetes.io/projected/c49252c7-3414-4786-b79f-bba3a3593934-kube-api-access-hhw2z\") pod \"maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:57.533537 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.533406 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c49252c7-3414-4786-b79f-bba3a3593934-maas-api-tls\") pod \"maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:57.533537 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.533445 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c49252c7-3414-4786-b79f-bba3a3593934-metrics-certs\") pod \"maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:57.533537 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.533501 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c49252c7-3414-4786-b79f-bba3a3593934-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:57.551259 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.551227 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:57.634549 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.634513 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhw2z\" (UniqueName: \"kubernetes.io/projected/c49252c7-3414-4786-b79f-bba3a3593934-kube-api-access-hhw2z\") pod \"maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:57.634703 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.634559 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c49252c7-3414-4786-b79f-bba3a3593934-maas-api-tls\") pod \"maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:57.634703 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.634583 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c49252c7-3414-4786-b79f-bba3a3593934-metrics-certs\") pod \"maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:57.634703 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.634613 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c49252c7-3414-4786-b79f-bba3a3593934-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:57.635172 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.635153 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c49252c7-3414-4786-b79f-bba3a3593934-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:57.637155 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.637129 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c49252c7-3414-4786-b79f-bba3a3593934-metrics-certs\") pod \"maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:57.637274 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.637211 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c49252c7-3414-4786-b79f-bba3a3593934-maas-api-tls\") pod \"maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:57.657171 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.657112 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhw2z\" (UniqueName: \"kubernetes.io/projected/c49252c7-3414-4786-b79f-bba3a3593934-kube-api-access-hhw2z\") pod \"maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:57.707582 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.707557 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:57.725462 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:57.725427 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:58.010455 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:58.010434 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6"] Oct 30 15:50:58.011087 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:50:58.011061 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc49252c7_3414_4786_b79f_bba3a3593934.slice/crio-7ccafcbae29466dc9cd4d7be64702f7b141a2d2ddc7d7297d934decc590b89dc WatchSource:0}: Error finding container 7ccafcbae29466dc9cd4d7be64702f7b141a2d2ddc7d7297d934decc590b89dc: Status 404 returned error can't find the container with id 7ccafcbae29466dc9cd4d7be64702f7b141a2d2ddc7d7297d934decc590b89dc Oct 30 15:50:58.472953 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:58.472924 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" event={"ID":"c49252c7-3414-4786-b79f-bba3a3593934","Type":"ContainerStarted","Data":"e016d46f9f2e98ed8515dd6b0dfa38ddd93de7ba8d6f8786c4291a17975ce38e"} Oct 30 15:50:58.473116 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:58.472971 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" event={"ID":"c49252c7-3414-4786-b79f-bba3a3593934","Type":"ContainerStarted","Data":"7ccafcbae29466dc9cd4d7be64702f7b141a2d2ddc7d7297d934decc590b89dc"} Oct 30 15:50:58.473116 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:58.473043 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:50:58.516763 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:58.516722 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" podStartSLOduration=1.212286371 podStartE2EDuration="1.516708975s" totalImagesPullingTime="304.422604ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:58.515088719 +0000 UTC m=+774.665985744" watchObservedRunningTime="2026-10-30 15:50:58.516708975 +0000 UTC m=+774.667606006" Oct 30 15:50:59.361387 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:59.361358 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:50:59.457987 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:50:59.457962 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 15:51:00.978063 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:00.978032 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm"] Oct 30 15:51:00.981897 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:00.981877 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:00.999500 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:00.999477 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-b24e7b-serving-cert\"" Oct 30 15:51:01.013159 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.013134 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm"] Oct 30 15:51:01.121577 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.121544 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:01.165853 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.165820 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:01.165999 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.165880 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-metrics-certs\") pod \"maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:01.166049 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.166010 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbsgm\" (UniqueName: \"kubernetes.io/projected/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-kube-api-access-lbsgm\") pod \"maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:01.166090 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.166047 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:01.266966 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.266879 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-metrics-certs\") pod \"maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:01.266966 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.266926 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbsgm\" (UniqueName: \"kubernetes.io/projected/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-kube-api-access-lbsgm\") pod \"maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:01.266966 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.266944 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:01.267241 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.266992 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:01.267241 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:51:01.267080 5013 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-serving-cert: secret "maas-api-e2e-worker-w4-b24e7b-serving-cert" not found Oct 30 15:51:01.267241 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:51:01.267140 5013 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-maas-api-tls podName:8821e53a-f78f-4139-ab3a-1cbafb6fe5a7 nodeName:}" failed. No retries permitted until 2026-10-30 15:51:01.767121231 +0000 UTC m=+777.918018257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-maas-api-tls") pod "maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" (UID: "8821e53a-f78f-4139-ab3a-1cbafb6fe5a7") : secret "maas-api-e2e-worker-w4-b24e7b-serving-cert" not found Oct 30 15:51:01.267973 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.267954 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:01.269440 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.269422 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-metrics-certs\") pod \"maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:01.286690 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.286649 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:01.291419 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.291395 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbsgm\" (UniqueName: \"kubernetes.io/projected/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-kube-api-access-lbsgm\") pod \"maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:01.460082 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.460049 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:01.774058 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.774022 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:01.776583 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.776555 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:01.894545 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:01.894508 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:02.207824 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:02.207798 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm"] Oct 30 15:51:02.210621 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:51:02.210577 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8821e53a_f78f_4139_ab3a_1cbafb6fe5a7.slice/crio-30b55ad8336038e864b12a6c21ee9bd496773b13a58e8c5c23d0b4a3ecae954e WatchSource:0}: Error finding container 30b55ad8336038e864b12a6c21ee9bd496773b13a58e8c5c23d0b4a3ecae954e: Status 404 returned error can't find the container with id 30b55ad8336038e864b12a6c21ee9bd496773b13a58e8c5c23d0b4a3ecae954e Oct 30 15:51:02.493414 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:02.493335 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" event={"ID":"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7","Type":"ContainerStarted","Data":"30b55ad8336038e864b12a6c21ee9bd496773b13a58e8c5c23d0b4a3ecae954e"} Oct 30 15:51:03.061318 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:03.061286 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:03.232031 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:03.232000 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:03.499203 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:03.499158 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" event={"ID":"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7","Type":"ContainerStarted","Data":"a00627808f8d474eca32e1d5f4f435e41b4c174d53f90e9f51be7f8ff99ce751"} Oct 30 15:51:03.499376 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:03.499246 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:03.574805 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:03.574745 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" podStartSLOduration=3.106660453 podStartE2EDuration="3.574733156s" totalImagesPullingTime="468.072703ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:03.574180318 +0000 UTC m=+779.725077350" watchObservedRunningTime="2026-10-30 15:51:03.574733156 +0000 UTC m=+779.725630184" Oct 30 15:51:04.484042 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:04.484012 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:51:04.795010 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:04.794918 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:05.195180 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:05.195154 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:05.426663 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:05.426632 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:05.608577 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:05.608480 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:05.783353 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:05.783322 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:07.053356 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:07.053327 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:07.320451 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:07.320375 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:09.204244 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:09.204177 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:09.510052 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:09.509975 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:51:10.067912 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.067877 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:10.671972 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.671937 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:10.746728 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.746701 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww"] Oct 30 15:51:10.752026 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.752006 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.777319 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.777291 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww"] Oct 30 15:51:10.848053 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.848028 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-b63c3c-openshift-default-dockercfg-94q4v\"" Oct 30 15:51:10.851143 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.851097 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:10.853322 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.853302 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.853406 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.853333 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6qwc\" (UniqueName: \"kubernetes.io/projected/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-kube-api-access-b6qwc\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.853406 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.853359 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.853524 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.853504 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.853557 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.853544 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.853659 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.853642 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.853722 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.853676 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.853775 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.853719 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.853831 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.853817 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.955320 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.955246 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.955473 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.955338 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.955473 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.955368 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6qwc\" (UniqueName: \"kubernetes.io/projected/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-kube-api-access-b6qwc\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.955473 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.955401 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.955473 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.955439 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.955473 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.955471 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.955731 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.955523 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.955731 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.955550 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.955731 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.955594 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.955946 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.955892 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.956067 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.955976 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.956067 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.956042 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.956172 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.956062 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.956280 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.956259 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.957868 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.957847 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.957951 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.957926 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.979252 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.979225 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:10.980885 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:10.980861 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6qwc\" (UniqueName: \"kubernetes.io/projected/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-kube-api-access-b6qwc\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:11.044740 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.044711 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:11.066355 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.066323 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:11.089210 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.089175 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z"] Oct 30 15:51:11.094925 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.094902 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.124186 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.123933 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z"] Oct 30 15:51:11.139514 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.139471 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-a2b3f1-openshift-default-dockercfg-jmzld\"" Oct 30 15:51:11.158720 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.158695 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.158850 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.158730 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.158850 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.158751 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.158918 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.158906 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.158953 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.158933 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.158987 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.158953 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.159022 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.158986 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.159022 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.159008 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.159089 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.159021 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhm7f\" (UniqueName: \"kubernetes.io/projected/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-kube-api-access-qhm7f\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.260316 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.260209 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.260316 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.260266 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.260590 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.260379 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhm7f\" (UniqueName: \"kubernetes.io/projected/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-kube-api-access-qhm7f\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.260590 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.260431 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.260590 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.260456 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.260590 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.260484 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.260812 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.260619 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.260812 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.260650 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.260812 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.260678 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.260812 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.260709 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.261120 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.260821 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.261239 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.261185 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.261791 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.261737 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.262075 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.262051 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.263939 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.263917 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.264068 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.264053 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.285244 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.285215 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.285417 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.285403 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhm7f\" (UniqueName: \"kubernetes.io/projected/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-kube-api-access-qhm7f\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.401328 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.401297 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww"] Oct 30 15:51:11.403453 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:51:11.403428 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d1b70a_d4d0_472d_b0fd_df159b4b3be3.slice/crio-8164bbe096f9221df24a9ef92f43afedcdca88dfdf498440fd0543d1ce82ed31 WatchSource:0}: Error finding container 8164bbe096f9221df24a9ef92f43afedcdca88dfdf498440fd0543d1ce82ed31: Status 404 returned error can't find the container with id 8164bbe096f9221df24a9ef92f43afedcdca88dfdf498440fd0543d1ce82ed31 Oct 30 15:51:11.405925 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.405890 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:11.406018 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.405961 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:11.406018 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.405994 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:11.416439 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.416417 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:11.541780 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.541055 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" event={"ID":"13d1b70a-d4d0-472d-b0fd-df159b4b3be3","Type":"ContainerStarted","Data":"9b02b5a5661a3133273423336faa9f56b5dbb96b16d09412399e048c9cdb02cd"} Oct 30 15:51:11.541780 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.541122 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" event={"ID":"13d1b70a-d4d0-472d-b0fd-df159b4b3be3","Type":"ContainerStarted","Data":"8164bbe096f9221df24a9ef92f43afedcdca88dfdf498440fd0543d1ce82ed31"} Oct 30 15:51:11.586595 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.586538 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" podStartSLOduration=1.586519848 podStartE2EDuration="1.586519848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:11.585266659 +0000 UTC m=+787.736163691" watchObservedRunningTime="2026-10-30 15:51:11.586519848 +0000 UTC m=+787.737416880" Oct 30 15:51:11.731752 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.731721 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z"] Oct 30 15:51:11.732703 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:51:11.732676 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea1e0986_aa76_4df4_85ac_cf104e42fb7e.slice/crio-ff593709e3754c83b83d2ef4a31fea8ca8854d7db5667a257872715e469a3bec WatchSource:0}: Error finding container ff593709e3754c83b83d2ef4a31fea8ca8854d7db5667a257872715e469a3bec: Status 404 returned error can't find the container with id ff593709e3754c83b83d2ef4a31fea8ca8854d7db5667a257872715e469a3bec Oct 30 15:51:11.735497 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.735382 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:11.735497 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.735476 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:11.735662 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:11.735535 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:12.067586 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:12.067496 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:12.068697 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:12.068672 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Oct 30 15:51:12.068821 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:12.068717 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" podUID="13d1b70a-d4d0-472d-b0fd-df159b4b3be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Oct 30 15:51:12.548335 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:12.548236 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" event={"ID":"ea1e0986-aa76-4df4-85ac-cf104e42fb7e","Type":"ContainerStarted","Data":"5acb53a976393e29bb46b7cb29feb5c459ab662969cb695d18258d6b43bcfe4f"} Oct 30 15:51:12.548335 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:12.548302 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" event={"ID":"ea1e0986-aa76-4df4-85ac-cf104e42fb7e","Type":"ContainerStarted","Data":"ff593709e3754c83b83d2ef4a31fea8ca8854d7db5667a257872715e469a3bec"} Oct 30 15:51:12.609227 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:12.603321 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" podStartSLOduration=1.6033040729999999 podStartE2EDuration="1.603304073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:12.602809657 +0000 UTC m=+788.753706690" watchObservedRunningTime="2026-10-30 15:51:12.603304073 +0000 UTC m=+788.754201105" Oct 30 15:51:13.067602 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:13.067569 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Oct 30 15:51:13.067964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:13.067654 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" podUID="13d1b70a-d4d0-472d-b0fd-df159b4b3be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Oct 30 15:51:13.266314 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:13.266278 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:13.419589 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:13.419513 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:13.421460 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:13.421256 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Oct 30 15:51:13.421460 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:13.421306 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" podUID="ea1e0986-aa76-4df4-85ac-cf104e42fb7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Oct 30 15:51:14.067248 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:14.067189 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Oct 30 15:51:14.067413 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:14.067274 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" podUID="13d1b70a-d4d0-472d-b0fd-df159b4b3be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Oct 30 15:51:14.423222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:14.420726 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Oct 30 15:51:14.423222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:14.420802 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" podUID="ea1e0986-aa76-4df4-85ac-cf104e42fb7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Oct 30 15:51:14.621881 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:14.621636 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:14.794478 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:14.794397 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:15.066899 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:15.066825 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Oct 30 15:51:15.066899 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:15.066877 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" podUID="13d1b70a-d4d0-472d-b0fd-df159b4b3be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Oct 30 15:51:15.309253 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:15.309224 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:15.421247 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:15.420827 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Oct 30 15:51:15.421247 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:15.420905 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" podUID="ea1e0986-aa76-4df4-85ac-cf104e42fb7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Oct 30 15:51:15.525302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:15.523609 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:15.763674 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:15.763604 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:16.067596 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:16.067505 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Oct 30 15:51:16.067596 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:16.067555 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" podUID="13d1b70a-d4d0-472d-b0fd-df159b4b3be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Oct 30 15:51:16.419723 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:16.419691 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Oct 30 15:51:16.420137 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:16.420093 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" podUID="ea1e0986-aa76-4df4-85ac-cf104e42fb7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Oct 30 15:51:17.067815 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:17.067775 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Oct 30 15:51:17.068227 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:17.067844 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" podUID="13d1b70a-d4d0-472d-b0fd-df159b4b3be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Oct 30 15:51:17.417390 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:17.417351 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Oct 30 15:51:17.417560 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:17.417439 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" podUID="ea1e0986-aa76-4df4-85ac-cf104e42fb7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Oct 30 15:51:17.712322 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:17.712150 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:18.071429 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:18.071351 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:18.071792 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:18.071744 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:18.072542 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:18.072519 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:51:18.421968 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:18.421936 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:18.422331 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:18.422313 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:18.423174 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:18.423153 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:51:19.481565 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:19.481522 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:20.322017 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:20.321986 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:20.602403 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:20.602330 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:20.823008 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:20.822977 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:21.028778 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:21.028748 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:21.264187 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:21.264157 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:21.477129 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:21.477098 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:21.638541 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:21.638510 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:23.031766 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.031737 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c"] Oct 30 15:51:23.035960 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.035937 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.053893 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.053868 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c"] Oct 30 15:51:23.063584 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.063539 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-ba9db9\"/\"default-dockercfg-zrw7s\"" Oct 30 15:51:23.063584 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.063539 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-ba9db9\"/\"openshift-service-ca.crt\"" Oct 30 15:51:23.063584 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.063575 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-ba9db9\"/\"kube-root-ca.crt\"" Oct 30 15:51:23.063812 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.063731 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-ba9db9\"/\"simulator-w5-ba9db9-kserve-self-signed-certs\"" Oct 30 15:51:23.146502 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.146473 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-model-cache\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.146689 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.146509 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-tmp-dir\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.146689 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.146537 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-dshm\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.146689 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.146554 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-home\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.146871 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.146746 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csshf\" (UniqueName: \"kubernetes.io/projected/c636d9c4-4262-46ba-9fd4-64388f70a905-kube-api-access-csshf\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.146871 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.146812 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c636d9c4-4262-46ba-9fd4-64388f70a905-tls-certs\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.194489 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.194457 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:23.247644 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.247613 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-dshm\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.247644 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.247643 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-home\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.247869 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.247689 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csshf\" (UniqueName: \"kubernetes.io/projected/c636d9c4-4262-46ba-9fd4-64388f70a905-kube-api-access-csshf\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.247869 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.247724 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c636d9c4-4262-46ba-9fd4-64388f70a905-tls-certs\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.247869 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.247811 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-model-cache\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.247869 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.247846 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-tmp-dir\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.248124 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.248104 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-home\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.248175 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.248120 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-model-cache\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.248175 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.248152 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-tmp-dir\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.249955 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.249929 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-dshm\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.250223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.250180 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c636d9c4-4262-46ba-9fd4-64388f70a905-tls-certs\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.270899 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.270878 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csshf\" (UniqueName: \"kubernetes.io/projected/c636d9c4-4262-46ba-9fd4-64388f70a905-kube-api-access-csshf\") pod \"simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.350880 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.350788 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:23.359456 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.359427 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:23.650233 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:23.650189 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c"] Oct 30 15:51:23.650683 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:51:23.650655 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc636d9c4_4262_46ba_9fd4_64388f70a905.slice/crio-28478d13f0a4cd675bfd926d07df35e11cd7dcfcec9ec6017b174a351cf06d3a WatchSource:0}: Error finding container 28478d13f0a4cd675bfd926d07df35e11cd7dcfcec9ec6017b174a351cf06d3a: Status 404 returned error can't find the container with id 28478d13f0a4cd675bfd926d07df35e11cd7dcfcec9ec6017b174a351cf06d3a Oct 30 15:51:24.454144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:24.454109 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:24.641982 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:24.641949 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" event={"ID":"c636d9c4-4262-46ba-9fd4-64388f70a905","Type":"ContainerStarted","Data":"28478d13f0a4cd675bfd926d07df35e11cd7dcfcec9ec6017b174a351cf06d3a"} Oct 30 15:51:25.196524 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:25.196482 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:25.648386 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:25.648353 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" event={"ID":"c636d9c4-4262-46ba-9fd4-64388f70a905","Type":"ContainerStarted","Data":"6d1fdff51d12f6704bb19a41d8b2a94e7f926bcde3b7fca9cbce469153650de3"} Oct 30 15:51:25.689810 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:25.689765 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podStartSLOduration=1.392006149 podStartE2EDuration="2.68975106s" totalImagesPullingTime="1.297744911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:25.68805514 +0000 UTC m=+801.838952171" watchObservedRunningTime="2026-10-30 15:51:25.68975106 +0000 UTC m=+801.840648108" Oct 30 15:51:25.951435 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:25.951290 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:26.351819 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:26.351741 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:26.585397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:26.585363 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:26.757096 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:26.757041 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:26.948374 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:26.948337 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:26.978758 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:51:26.978716 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:51:28.863468 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:28.863435 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:30.492846 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:30.492810 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:32.200504 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:32.200464 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:33.285652 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:33.285578 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:33.351441 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:33.351404 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:33.364301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:33.364272 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:33.704002 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:33.703977 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:51:34.097516 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:34.097414 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:35.748962 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:35.748933 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:35.916597 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:35.916564 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:37.475602 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:37.475526 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:39.308369 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:39.308338 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:40.813460 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:40.813429 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6"] Oct 30 15:51:40.818635 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:40.818610 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:40.847878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:40.847854 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6"] Oct 30 15:51:40.922586 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:40.922554 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:40.922740 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:40.922595 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:40.922740 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:40.922674 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:40.922740 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:40.922706 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:40.922916 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:40.922753 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrmwx\" (UniqueName: \"kubernetes.io/projected/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-kube-api-access-qrmwx\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:40.922916 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:40.922821 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:40.922916 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:40.922842 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:40.922916 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:40.922907 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:40.923112 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:40.922930 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.023833 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.023796 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.024002 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.023846 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.024002 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.023924 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.024002 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.023965 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.024170 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.024005 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.024170 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.024032 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.024170 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.024066 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrmwx\" (UniqueName: \"kubernetes.io/projected/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-kube-api-access-qrmwx\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.024170 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.024103 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.024170 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.024127 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.024444 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.024255 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.024444 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.024374 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.024575 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.024552 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.024667 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.024645 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.024731 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.024653 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.026750 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.026725 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.026999 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.026979 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.048152 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.048129 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.048491 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.048474 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrmwx\" (UniqueName: \"kubernetes.io/projected/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-kube-api-access-qrmwx\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.086851 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.086767 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:41.133661 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.133629 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:41.176691 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.176658 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv"] Oct 30 15:51:41.185118 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.183620 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.221552 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.221524 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv"] Oct 30 15:51:41.226829 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.226799 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52a418d-1057-4580-8728-3fe24c4d308d-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.226945 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.226871 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djjnf\" (UniqueName: \"kubernetes.io/projected/f52a418d-1057-4580-8728-3fe24c4d308d-kube-api-access-djjnf\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.227026 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.226947 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.227026 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.226988 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.227026 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.227020 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52a418d-1057-4580-8728-3fe24c4d308d-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.227185 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.227062 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.227185 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.227102 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.227185 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.227134 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.227185 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.227179 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52a418d-1057-4580-8728-3fe24c4d308d-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.328231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.328177 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.328406 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.328246 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.328406 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.328271 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52a418d-1057-4580-8728-3fe24c4d308d-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.328406 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.328304 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.328406 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.328335 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.328665 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.328448 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.328665 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.328543 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52a418d-1057-4580-8728-3fe24c4d308d-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.328665 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.328599 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52a418d-1057-4580-8728-3fe24c4d308d-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.328923 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.328672 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.328923 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.328683 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djjnf\" (UniqueName: \"kubernetes.io/projected/f52a418d-1057-4580-8728-3fe24c4d308d-kube-api-access-djjnf\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.328923 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.328767 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.329080 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.329063 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52a418d-1057-4580-8728-3fe24c4d308d-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.329167 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.329141 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.329532 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.329508 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.330868 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.330852 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.331158 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.331141 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52a418d-1057-4580-8728-3fe24c4d308d-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.351853 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.351793 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52a418d-1057-4580-8728-3fe24c4d308d-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.352075 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.352055 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djjnf\" (UniqueName: \"kubernetes.io/projected/f52a418d-1057-4580-8728-3fe24c4d308d-kube-api-access-djjnf\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.471856 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.471832 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6"] Oct 30 15:51:41.475423 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:51:41.475395 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f1ad3ab_f617_4a25_a2c3_c47ba625624f.slice/crio-a3ea535cc9cf68316e1a552c6f7bac3a6eaaa0232204c459a36fa544f88d4a7f WatchSource:0}: Error finding container a3ea535cc9cf68316e1a552c6f7bac3a6eaaa0232204c459a36fa544f88d4a7f: Status 404 returned error can't find the container with id a3ea535cc9cf68316e1a552c6f7bac3a6eaaa0232204c459a36fa544f88d4a7f Oct 30 15:51:41.477461 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.477431 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:41.477525 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.477498 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:41.477566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.477530 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:41.524079 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.524060 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:41.735481 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.735445 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" event={"ID":"6f1ad3ab-f617-4a25-a2c3-c47ba625624f","Type":"ContainerStarted","Data":"e034790e8f537c1b3e443c50433aa934d1df5b7c06c5052d44a1de26423c0e7e"} Oct 30 15:51:41.735672 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.735504 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" event={"ID":"6f1ad3ab-f617-4a25-a2c3-c47ba625624f","Type":"ContainerStarted","Data":"a3ea535cc9cf68316e1a552c6f7bac3a6eaaa0232204c459a36fa544f88d4a7f"} Oct 30 15:51:41.780261 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.780168 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" podStartSLOduration=1.780149872 podStartE2EDuration="1.780149872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:41.779610006 +0000 UTC m=+817.930507050" watchObservedRunningTime="2026-10-30 15:51:41.780149872 +0000 UTC m=+817.931046904" Oct 30 15:51:41.845779 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.845746 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv"] Oct 30 15:51:41.846635 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:51:41.846606 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf52a418d_1057_4580_8728_3fe24c4d308d.slice/crio-dee1ea9dc80bf9cb97fbdd129031396ca99c19b7feaec84833bdfaedff56d5d2 WatchSource:0}: Error finding container dee1ea9dc80bf9cb97fbdd129031396ca99c19b7feaec84833bdfaedff56d5d2: Status 404 returned error can't find the container with id dee1ea9dc80bf9cb97fbdd129031396ca99c19b7feaec84833bdfaedff56d5d2 Oct 30 15:51:41.849039 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.849009 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:41.849134 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.849074 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:41.849134 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:41.849106 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:42.134318 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:42.134290 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:42.135876 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:42.135829 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 30 15:51:42.136018 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:42.135893 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" podUID="6f1ad3ab-f617-4a25-a2c3-c47ba625624f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 30 15:51:42.744007 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:42.743981 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" event={"ID":"f52a418d-1057-4580-8728-3fe24c4d308d","Type":"ContainerStarted","Data":"d981588a7958e71322c571d89ad2070f1572720119846fcb3cc280fc8c21c59b"} Oct 30 15:51:42.744303 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:42.744263 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" event={"ID":"f52a418d-1057-4580-8728-3fe24c4d308d","Type":"ContainerStarted","Data":"dee1ea9dc80bf9cb97fbdd129031396ca99c19b7feaec84833bdfaedff56d5d2"} Oct 30 15:51:42.801160 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:42.801082 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" podStartSLOduration=1.801062494 podStartE2EDuration="1.801062494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:42.798884639 +0000 UTC m=+818.949781673" watchObservedRunningTime="2026-10-30 15:51:42.801062494 +0000 UTC m=+818.951959527" Oct 30 15:51:43.001240 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:43.001125 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:43.134747 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:43.134703 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 30 15:51:43.134926 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:43.134756 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" podUID="6f1ad3ab-f617-4a25-a2c3-c47ba625624f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 30 15:51:43.525298 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:43.525258 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:43.526289 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:43.526251 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Oct 30 15:51:43.526412 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:43.526304 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" podUID="f52a418d-1057-4580-8728-3fe24c4d308d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Oct 30 15:51:44.134835 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.134803 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 30 15:51:44.135287 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.135261 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" podUID="6f1ad3ab-f617-4a25-a2c3-c47ba625624f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 30 15:51:44.327266 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.308775 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp"] Oct 30 15:51:44.333385 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.333358 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.335625 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.335595 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp"] Oct 30 15:51:44.360218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.360180 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-c129d1\"/\"kube-root-ca.crt\"" Oct 30 15:51:44.360348 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.360227 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-c129d1\"/\"openshift-service-ca.crt\"" Oct 30 15:51:44.360412 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.360352 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-c129d1\"/\"default-dockercfg-kc6fq\"" Oct 30 15:51:44.360412 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.360363 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-c129d1\"/\"simulator-w6-c129d1-kserve-self-signed-certs\"" Oct 30 15:51:44.363871 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.363851 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-dshm\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.364037 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.364019 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae0e7089-01f3-42b4-9590-51cdc0b4765a-tls-certs\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.364116 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.364046 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-model-cache\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.364177 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.364165 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-home\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.364253 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.364242 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnx6p\" (UniqueName: \"kubernetes.io/projected/ae0e7089-01f3-42b4-9590-51cdc0b4765a-kube-api-access-bnx6p\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.364316 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.364302 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-tmp-dir\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.430273 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.430111 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:44.465643 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.465607 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-dshm\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.465643 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.465645 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae0e7089-01f3-42b4-9590-51cdc0b4765a-tls-certs\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.465878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.465671 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-model-cache\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.465878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.465773 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-home\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.465878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.465857 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnx6p\" (UniqueName: \"kubernetes.io/projected/ae0e7089-01f3-42b4-9590-51cdc0b4765a-kube-api-access-bnx6p\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.466038 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.465912 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-tmp-dir\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.466311 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.466287 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-tmp-dir\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.466457 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.466440 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-model-cache\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.466559 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.466458 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-home\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.468578 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.468553 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-dshm\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.468922 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.468905 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae0e7089-01f3-42b4-9590-51cdc0b4765a-tls-certs\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.528815 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.526759 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Oct 30 15:51:44.528815 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.526817 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" podUID="f52a418d-1057-4580-8728-3fe24c4d308d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Oct 30 15:51:44.561983 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.561946 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnx6p\" (UniqueName: \"kubernetes.io/projected/ae0e7089-01f3-42b4-9590-51cdc0b4765a-kube-api-access-bnx6p\") pod \"simulator-w6-c129d1-kserve-b65d8b855-5h8dp\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:44.650596 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:44.650561 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:45.051824 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:45.051644 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp"] Oct 30 15:51:45.054562 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:51:45.054514 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae0e7089_01f3_42b4_9590_51cdc0b4765a.slice/crio-4dfb219ce03d7a6a88cf48f7293262080c9dbc2d44d1372f2c51e064cfb4ce00 WatchSource:0}: Error finding container 4dfb219ce03d7a6a88cf48f7293262080c9dbc2d44d1372f2c51e064cfb4ce00: Status 404 returned error can't find the container with id 4dfb219ce03d7a6a88cf48f7293262080c9dbc2d44d1372f2c51e064cfb4ce00 Oct 30 15:51:45.134364 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:45.134328 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 30 15:51:45.134563 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:45.134386 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" podUID="6f1ad3ab-f617-4a25-a2c3-c47ba625624f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 30 15:51:45.231918 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:45.231884 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:45.524671 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:45.524621 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Oct 30 15:51:45.524885 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:45.524686 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" podUID="f52a418d-1057-4580-8728-3fe24c4d308d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Oct 30 15:51:45.805273 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:45.790574 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" event={"ID":"ae0e7089-01f3-42b4-9590-51cdc0b4765a","Type":"ContainerStarted","Data":"ecab5337e9406a8bb30208eb2473a9abf402401dbdbf0f53014329966f8281dc"} Oct 30 15:51:45.805273 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:45.790632 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" event={"ID":"ae0e7089-01f3-42b4-9590-51cdc0b4765a","Type":"ContainerStarted","Data":"4dfb219ce03d7a6a88cf48f7293262080c9dbc2d44d1372f2c51e064cfb4ce00"} Oct 30 15:51:45.849214 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:45.849143 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podStartSLOduration=1.849129104 podStartE2EDuration="1.849129104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:45.848388937 +0000 UTC m=+821.999285965" watchObservedRunningTime="2026-10-30 15:51:45.849129104 +0000 UTC m=+822.000026134" Oct 30 15:51:46.134724 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:46.134693 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 30 15:51:46.134948 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:46.134925 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" podUID="6f1ad3ab-f617-4a25-a2c3-c47ba625624f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 30 15:51:46.525117 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:46.525029 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Oct 30 15:51:46.525604 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:46.525288 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" podUID="f52a418d-1057-4580-8728-3fe24c4d308d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Oct 30 15:51:46.651023 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:46.650977 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:47.134916 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:47.134875 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 30 15:51:47.135116 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:47.134943 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" podUID="6f1ad3ab-f617-4a25-a2c3-c47ba625624f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 30 15:51:47.292294 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:47.292165 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:47.525449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:47.525364 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Oct 30 15:51:47.525449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:47.525434 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" podUID="f52a418d-1057-4580-8728-3fe24c4d308d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Oct 30 15:51:48.139282 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:48.139252 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:48.139668 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:48.139644 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:48.140627 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:48.140609 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:51:48.528782 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:48.528714 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:48.529164 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:48.528973 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:48.529571 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:48.529553 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:51:49.243510 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:49.243454 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:51.090459 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:51.090427 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:52.703736 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.703705 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:52.848263 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.848236 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n"] Oct 30 15:51:52.852313 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.852292 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:52.880099 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.880072 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-040f9a\"/\"premium-w3-040f9a-kserve-self-signed-certs\"" Oct 30 15:51:52.880300 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.880074 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-040f9a\"/\"openshift-service-ca.crt\"" Oct 30 15:51:52.880300 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.880104 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-040f9a\"/\"kube-root-ca.crt\"" Oct 30 15:51:52.880487 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.880281 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-040f9a\"/\"default-dockercfg-7zh4d\"" Oct 30 15:51:52.882451 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.882421 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n"] Oct 30 15:51:52.910432 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.910410 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjxq2\" (UniqueName: \"kubernetes.io/projected/dd53a00a-b214-47dd-8a64-af52d0e490a5-kube-api-access-vjxq2\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:52.910530 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.910447 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd53a00a-b214-47dd-8a64-af52d0e490a5-tls-certs\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:52.910530 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.910471 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-tmp-dir\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:52.910637 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.910620 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-dshm\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:52.910673 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.910649 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-model-cache\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:52.910673 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.910667 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-home\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:52.981808 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:52.981731 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:53.012427 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.012389 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd53a00a-b214-47dd-8a64-af52d0e490a5-tls-certs\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:53.012583 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.012450 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-tmp-dir\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:53.012583 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.012546 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-dshm\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:53.012583 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.012578 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-model-cache\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:53.012729 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.012605 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-home\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:53.012729 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.012713 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjxq2\" (UniqueName: \"kubernetes.io/projected/dd53a00a-b214-47dd-8a64-af52d0e490a5-kube-api-access-vjxq2\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:53.012932 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.012901 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-tmp-dir\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:53.013104 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.013086 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-home\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:53.013223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.012956 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-model-cache\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:53.015101 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.015044 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-dshm\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:53.015694 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.015661 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd53a00a-b214-47dd-8a64-af52d0e490a5-tls-certs\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:53.042789 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.042759 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjxq2\" (UniqueName: \"kubernetes.io/projected/dd53a00a-b214-47dd-8a64-af52d0e490a5-kube-api-access-vjxq2\") pod \"premium-w3-040f9a-kserve-5997dfdf79-vzk2n\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:53.166544 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.166515 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:53.474064 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.474040 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n"] Oct 30 15:51:53.474248 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:51:53.474227 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd53a00a_b214_47dd_8a64_af52d0e490a5.slice/crio-b53a0e2e90b3c4bcd14d7d0ce64e42de48d2f2f4a156c9f5fa87786ae89d40b8 WatchSource:0}: Error finding container b53a0e2e90b3c4bcd14d7d0ce64e42de48d2f2f4a156c9f5fa87786ae89d40b8: Status 404 returned error can't find the container with id b53a0e2e90b3c4bcd14d7d0ce64e42de48d2f2f4a156c9f5fa87786ae89d40b8 Oct 30 15:51:53.869649 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.869560 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" event={"ID":"dd53a00a-b214-47dd-8a64-af52d0e490a5","Type":"ContainerStarted","Data":"76d3bd542e85243573a8287f44449eade4b06560050231f2a5c20bd4469ed877"} Oct 30 15:51:53.869649 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.869603 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" event={"ID":"dd53a00a-b214-47dd-8a64-af52d0e490a5","Type":"ContainerStarted","Data":"b53a0e2e90b3c4bcd14d7d0ce64e42de48d2f2f4a156c9f5fa87786ae89d40b8"} Oct 30 15:51:53.910985 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:53.910932 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podStartSLOduration=1.910918798 podStartE2EDuration="1.910918798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:53.909605363 +0000 UTC m=+830.060502395" watchObservedRunningTime="2026-10-30 15:51:53.910918798 +0000 UTC m=+830.061815829" Oct 30 15:51:54.166765 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:54.166735 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:51:54.571838 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:54.571756 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:54.650754 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:54.650721 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:54.663126 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:54.663103 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:54.822823 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:54.822736 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:54.889331 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:54.889304 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:51:55.847770 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:55.847737 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9"] Oct 30 15:51:55.851697 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:55.851675 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:55.881477 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:55.881449 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9"] Oct 30 15:51:55.941137 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:55.941108 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:55.941567 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:55.941140 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:55.941567 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:55.941173 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:55.941567 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:55.941262 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04fbe630-342a-4ec9-91c6-48c89165b6de-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:55.941567 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:55.941324 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:55.941567 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:55.941350 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:55.941567 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:55.941427 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:55.941567 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:55.941499 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scxr7\" (UniqueName: \"kubernetes.io/projected/04fbe630-342a-4ec9-91c6-48c89165b6de-kube-api-access-scxr7\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:55.941567 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:55.941540 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.043148 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.043119 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.043148 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.043152 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.043430 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.043171 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.043430 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.043186 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04fbe630-342a-4ec9-91c6-48c89165b6de-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.043430 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.043227 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.043430 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.043249 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.043430 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.043276 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.043430 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.043310 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scxr7\" (UniqueName: \"kubernetes.io/projected/04fbe630-342a-4ec9-91c6-48c89165b6de-kube-api-access-scxr7\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.043430 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.043356 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.043783 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.043745 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.043848 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.043788 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.043893 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.043852 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.043893 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.043865 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.044052 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.043998 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04fbe630-342a-4ec9-91c6-48c89165b6de-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.045795 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.045770 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.046144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.046125 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.065990 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.065967 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.066089 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.066071 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scxr7\" (UniqueName: \"kubernetes.io/projected/04fbe630-342a-4ec9-91c6-48c89165b6de-kube-api-access-scxr7\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.165714 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.165684 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:56.217381 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.217354 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46"] Oct 30 15:51:56.222454 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.221658 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.250525 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.249650 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46"] Oct 30 15:51:56.347117 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.347084 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.347292 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.347129 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.347292 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.347273 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.347369 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.347301 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.347420 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.347407 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.347458 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.347439 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n87fs\" (UniqueName: \"kubernetes.io/projected/4e6a88f2-c53c-4346-81b6-68ea50a6528a-kube-api-access-n87fs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.347489 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.347464 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.347558 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.347542 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.347596 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.347572 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.449096 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.449028 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.449096 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.449071 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.449326 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.449134 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.449326 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.449171 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n87fs\" (UniqueName: \"kubernetes.io/projected/4e6a88f2-c53c-4346-81b6-68ea50a6528a-kube-api-access-n87fs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.449326 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.449227 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.449326 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.449270 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.449326 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.449299 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.449660 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.449338 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.449660 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.449391 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.449660 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.449464 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.449660 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.449564 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.449660 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.449621 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.449922 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.449861 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.449922 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.449885 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.451869 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.451848 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.452035 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.452018 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.471872 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.471843 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.471941 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.471924 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n87fs\" (UniqueName: \"kubernetes.io/projected/4e6a88f2-c53c-4346-81b6-68ea50a6528a-kube-api-access-n87fs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.488573 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.488376 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9"] Oct 30 15:51:56.489519 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:51:56.489494 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04fbe630_342a_4ec9_91c6_48c89165b6de.slice/crio-a913cec80bc481b4875bbdfbb94090495fc9676c6457d1ffcd19551958749d1c WatchSource:0}: Error finding container a913cec80bc481b4875bbdfbb94090495fc9676c6457d1ffcd19551958749d1c: Status 404 returned error can't find the container with id a913cec80bc481b4875bbdfbb94090495fc9676c6457d1ffcd19551958749d1c Oct 30 15:51:56.491673 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.491640 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:56.492119 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.492091 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:56.492170 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.492153 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:56.492290 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.492185 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:56.553271 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.553248 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:56.875097 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.875068 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46"] Oct 30 15:51:56.875909 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:51:56.875879 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e6a88f2_c53c_4346_81b6_68ea50a6528a.slice/crio-87bdc85aa4cf9d98954cd31d38debc8e7d14b1a917e63b7e4597e6a19edbade5 WatchSource:0}: Error finding container 87bdc85aa4cf9d98954cd31d38debc8e7d14b1a917e63b7e4597e6a19edbade5: Status 404 returned error can't find the container with id 87bdc85aa4cf9d98954cd31d38debc8e7d14b1a917e63b7e4597e6a19edbade5 Oct 30 15:51:56.878433 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.878397 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:56.878541 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.878473 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:56.878541 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.878519 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:51:56.893843 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.892511 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" event={"ID":"04fbe630-342a-4ec9-91c6-48c89165b6de","Type":"ContainerStarted","Data":"3f41587f23dc811b0acffaf42eeb5e5124c10777aa08a7dd37bbeddd6fbee0ab"} Oct 30 15:51:56.893843 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.892574 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" event={"ID":"04fbe630-342a-4ec9-91c6-48c89165b6de","Type":"ContainerStarted","Data":"a913cec80bc481b4875bbdfbb94090495fc9676c6457d1ffcd19551958749d1c"} Oct 30 15:51:56.894447 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.893937 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" event={"ID":"4e6a88f2-c53c-4346-81b6-68ea50a6528a","Type":"ContainerStarted","Data":"87bdc85aa4cf9d98954cd31d38debc8e7d14b1a917e63b7e4597e6a19edbade5"} Oct 30 15:51:56.937293 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:56.937251 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" podStartSLOduration=1.937234212 podStartE2EDuration="1.937234212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:56.936113229 +0000 UTC m=+833.087010272" watchObservedRunningTime="2026-10-30 15:51:56.937234212 +0000 UTC m=+833.088131241" Oct 30 15:51:57.010907 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:51:57.010862 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:51:57.166578 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:57.166535 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:51:57.167877 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:57.167846 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 30 15:51:57.167989 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:57.167899 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" podUID="04fbe630-342a-4ec9-91c6-48c89165b6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 30 15:51:57.915221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:57.914839 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" event={"ID":"4e6a88f2-c53c-4346-81b6-68ea50a6528a","Type":"ContainerStarted","Data":"88a1e60d71eee58e70ec295441da04abb42fe59757e033ea0a4d8b5a646513fe"} Oct 30 15:51:57.974222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:57.970685 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" podStartSLOduration=1.9706685849999999 podStartE2EDuration="1.970668585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:57.969522795 +0000 UTC m=+834.120419826" watchObservedRunningTime="2026-10-30 15:51:57.970668585 +0000 UTC m=+834.121565617" Oct 30 15:51:58.166693 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:58.166612 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 30 15:51:58.166693 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:58.166667 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" podUID="04fbe630-342a-4ec9-91c6-48c89165b6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 30 15:51:58.309223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:58.301608 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:51:58.553529 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:58.553445 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:51:58.554807 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:58.554785 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Oct 30 15:51:58.554910 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:58.554838 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" podUID="4e6a88f2-c53c-4346-81b6-68ea50a6528a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Oct 30 15:51:59.166585 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:59.166555 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 30 15:51:59.166815 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:59.166790 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" podUID="04fbe630-342a-4ec9-91c6-48c89165b6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 30 15:51:59.553979 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:59.553891 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Oct 30 15:51:59.554258 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:51:59.554227 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" podUID="4e6a88f2-c53c-4346-81b6-68ea50a6528a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Oct 30 15:52:00.166540 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:00.166504 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 30 15:52:00.166738 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:00.166564 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" podUID="04fbe630-342a-4ec9-91c6-48c89165b6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 30 15:52:00.382527 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:00.380007 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:00.553776 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:00.553697 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Oct 30 15:52:00.553776 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:00.553750 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" podUID="4e6a88f2-c53c-4346-81b6-68ea50a6528a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Oct 30 15:52:01.166464 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:01.166427 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 30 15:52:01.166633 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:01.166483 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" podUID="04fbe630-342a-4ec9-91c6-48c89165b6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 30 15:52:01.554001 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:01.553923 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Oct 30 15:52:01.554001 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:01.553976 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" podUID="4e6a88f2-c53c-4346-81b6-68ea50a6528a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Oct 30 15:52:02.166385 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:02.166350 5013 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 30 15:52:02.166549 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:02.166397 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" podUID="04fbe630-342a-4ec9-91c6-48c89165b6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 30 15:52:02.345315 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:02.345287 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:02.553977 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:02.553884 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Oct 30 15:52:02.553977 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:02.553946 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" podUID="4e6a88f2-c53c-4346-81b6-68ea50a6528a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Oct 30 15:52:03.166719 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:03.166690 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:52:03.211058 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:03.211029 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:52:03.219628 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:03.219598 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:52:03.219957 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:03.219922 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:52:03.222161 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:03.222134 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:52:03.553569 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:03.553488 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Oct 30 15:52:03.553569 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:03.553539 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" podUID="4e6a88f2-c53c-4346-81b6-68ea50a6528a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Oct 30 15:52:03.939154 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:03.939123 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:03.984032 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:03.984005 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:52:04.195682 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:04.195560 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:04.558353 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:04.558284 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:52:04.558583 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:04.558562 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:52:04.559298 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:04.559281 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:52:06.088231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:06.088165 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:07.636047 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:07.636018 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:07.800838 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:07.800774 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:09.488951 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:09.488920 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:09.912884 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:09.912849 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:11.164142 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:11.164111 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:13.131795 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:13.131761 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:15.096121 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.096023 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:15.335157 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.335124 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f"] Oct 30 15:52:15.342869 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.342835 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.360628 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.360595 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f"] Oct 30 15:52:15.369032 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.369002 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-a2b3f1\"/\"openshift-service-ca.crt\"" Oct 30 15:52:15.369188 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.369060 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-a2b3f1\"/\"kube-root-ca.crt\"" Oct 30 15:52:15.369188 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.369095 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-a2b3f1\"/\"default-dockercfg-gvgcj\"" Oct 30 15:52:15.369188 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.369132 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-a2b3f1\"/\"premium-w1-a2b3f1-kserve-self-signed-certs\"" Oct 30 15:52:15.423673 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.423646 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-home\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.423806 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.423714 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-tmp-dir\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.423869 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.423822 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4lzm\" (UniqueName: \"kubernetes.io/projected/82fbbdd5-fc1a-421a-9bc2-865304cac190-kube-api-access-x4lzm\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.423917 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.423868 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-dshm\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.423968 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.423932 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82fbbdd5-fc1a-421a-9bc2-865304cac190-tls-certs\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.423968 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.423953 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-model-cache\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.460461 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.460428 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:15.525560 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.525526 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4lzm\" (UniqueName: \"kubernetes.io/projected/82fbbdd5-fc1a-421a-9bc2-865304cac190-kube-api-access-x4lzm\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.525724 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.525574 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-dshm\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.525724 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.525624 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82fbbdd5-fc1a-421a-9bc2-865304cac190-tls-certs\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.525724 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.525642 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-model-cache\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.525724 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.525725 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-home\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.525941 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.525775 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-tmp-dir\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.526534 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.526507 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-tmp-dir\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.526644 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.526533 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-home\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.526644 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.526631 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-model-cache\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.528252 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.528232 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82fbbdd5-fc1a-421a-9bc2-865304cac190-tls-certs\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.528401 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.528386 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-dshm\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.551357 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.551322 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4lzm\" (UniqueName: \"kubernetes.io/projected/82fbbdd5-fc1a-421a-9bc2-865304cac190-kube-api-access-x4lzm\") pod \"premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.660597 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.660562 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:15.964685 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:15.964661 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f"] Oct 30 15:52:15.965417 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:52:15.965395 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82fbbdd5_fc1a_421a_9bc2_865304cac190.slice/crio-a5aecf3c795ee410b3e62bab2b26f7a17da886155084f47a0dce679cc628e2cc WatchSource:0}: Error finding container a5aecf3c795ee410b3e62bab2b26f7a17da886155084f47a0dce679cc628e2cc: Status 404 returned error can't find the container with id a5aecf3c795ee410b3e62bab2b26f7a17da886155084f47a0dce679cc628e2cc Oct 30 15:52:16.048915 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:16.048886 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" event={"ID":"82fbbdd5-fc1a-421a-9bc2-865304cac190","Type":"ContainerStarted","Data":"6b327307c5e16e4f4124a4c68a635f2a0ea3da6aeeabe720cd0f3c70befa9c46"} Oct 30 15:52:16.049041 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:16.048934 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" event={"ID":"82fbbdd5-fc1a-421a-9bc2-865304cac190","Type":"ContainerStarted","Data":"a5aecf3c795ee410b3e62bab2b26f7a17da886155084f47a0dce679cc628e2cc"} Oct 30 15:52:16.088461 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:16.088416 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podStartSLOduration=1.088401958 podStartE2EDuration="1.088401958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:52:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:52:16.08815788 +0000 UTC m=+852.239054907" watchObservedRunningTime="2026-10-30 15:52:16.088401958 +0000 UTC m=+852.239298999" Oct 30 15:52:16.660938 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:16.660905 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:16.751536 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:16.751487 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:16.918269 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:16.918153 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:18.586032 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:18.585984 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:19.991949 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:19.991916 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:20.306438 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:20.306331 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:22.012568 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:22.012532 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:23.887215 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:23.886608 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:25.624295 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:25.624267 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:25.661651 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:25.660946 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:25.675518 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:25.675493 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:26.111431 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.111390 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t"] Oct 30 15:52:26.120260 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.120224 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.142522 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.142489 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-ba9db9-openshift-default-dockercfg-lczk5\"" Oct 30 15:52:26.147629 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.146911 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t"] Oct 30 15:52:26.307219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.275353 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.307219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.275420 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.307219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.275495 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.307219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.275523 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91ed8818-5236-42e2-92f5-74e29232e3e5-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.307219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.275553 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmw7h\" (UniqueName: \"kubernetes.io/projected/91ed8818-5236-42e2-92f5-74e29232e3e5-kube-api-access-fmw7h\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.307219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.275591 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.307219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.275670 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.307219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.275731 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.307219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.275758 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.365604 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.365520 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:52:26.379809 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.379775 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.380029 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.380010 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.380184 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.380166 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.394404 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.394382 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91ed8818-5236-42e2-92f5-74e29232e3e5-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.394532 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.394516 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmw7h\" (UniqueName: \"kubernetes.io/projected/91ed8818-5236-42e2-92f5-74e29232e3e5-kube-api-access-fmw7h\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.394658 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.394643 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.394813 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.394794 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.394970 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.394954 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.395082 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.395067 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.405224 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.404792 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91ed8818-5236-42e2-92f5-74e29232e3e5-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.405224 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.404922 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.405224 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.405025 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.405640 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.405591 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.405865 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.405846 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.435474 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.435445 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.442356 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.440887 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmw7h\" (UniqueName: \"kubernetes.io/projected/91ed8818-5236-42e2-92f5-74e29232e3e5-kube-api-access-fmw7h\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.447282 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.432629 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.565220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.564108 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.609218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:26.608487 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:26.980895 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:52:26.980776 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:52:27.111869 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:27.111832 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t"] Oct 30 15:52:27.112497 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:52:27.112469 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91ed8818_5236_42e2_92f5_74e29232e3e5.slice/crio-7b59e537fd8adfd6822797108cda446b4c1d17b8999597b6393c70599d79e58b WatchSource:0}: Error finding container 7b59e537fd8adfd6822797108cda446b4c1d17b8999597b6393c70599d79e58b: Status 404 returned error can't find the container with id 7b59e537fd8adfd6822797108cda446b4c1d17b8999597b6393c70599d79e58b Oct 30 15:52:27.114867 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:27.114833 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:52:27.114968 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:27.114908 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:52:27.115031 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:27.114976 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:52:27.122898 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:27.122830 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" event={"ID":"91ed8818-5236-42e2-92f5-74e29232e3e5","Type":"ContainerStarted","Data":"7b59e537fd8adfd6822797108cda446b4c1d17b8999597b6393c70599d79e58b"} Oct 30 15:52:27.697794 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:27.697736 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:27.941478 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:27.941441 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:28.131612 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:28.131576 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" event={"ID":"91ed8818-5236-42e2-92f5-74e29232e3e5","Type":"ContainerStarted","Data":"8761b289dee89aa1f9575cbf1c5e82dc33d096463fe3d8abd63359749176ab90"} Oct 30 15:52:28.190086 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:28.189845 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" podStartSLOduration=2.189814056 podStartE2EDuration="2.189814056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:52:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:52:28.189240259 +0000 UTC m=+864.340137292" watchObservedRunningTime="2026-10-30 15:52:28.189814056 +0000 UTC m=+864.340711089" Oct 30 15:52:28.609669 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:28.609589 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:28.611046 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:28.611021 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 15:52:28.611166 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:28.611067 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" podUID="91ed8818-5236-42e2-92f5-74e29232e3e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 15:52:29.503216 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:29.499123 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:29.609819 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:29.609787 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 15:52:29.609986 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:29.609841 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" podUID="91ed8818-5236-42e2-92f5-74e29232e3e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 15:52:30.609070 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:30.609042 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 15:52:30.609515 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:30.609491 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" podUID="91ed8818-5236-42e2-92f5-74e29232e3e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 15:52:31.509843 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:31.509811 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:31.609746 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:31.609704 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 15:52:31.610128 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:31.609769 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" podUID="91ed8818-5236-42e2-92f5-74e29232e3e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 15:52:32.609315 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:32.609285 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 15:52:32.609544 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:32.609522 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" podUID="91ed8818-5236-42e2-92f5-74e29232e3e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 15:52:33.281831 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:33.281796 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:33.609473 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:33.609398 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 15:52:33.609473 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:33.609458 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" podUID="91ed8818-5236-42e2-92f5-74e29232e3e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 15:52:34.613659 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:34.613629 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:34.614135 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:34.614033 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:34.614689 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:34.614671 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 15:52:35.019678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:35.019638 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:36.904606 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:36.904573 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:38.627628 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:38.627585 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:40.476979 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:40.476942 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:40.765891 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:40.765818 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5"] Oct 30 15:52:40.769995 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:40.769978 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:40.791541 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:40.791510 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5"] Oct 30 15:52:40.879598 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:40.879552 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-040f9a-openshift-default-dockercfg-r5zts\"" Oct 30 15:52:40.915950 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:40.915921 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-envoy\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:40.916075 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:40.915976 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-podinfo\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:40.916075 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:40.916045 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-workload-socket\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:40.916171 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:40.916119 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0f87920-2518-4aa2-9db4-5959b7e71a54-istiod-ca-cert\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:40.916248 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:40.916187 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-credential-socket\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:40.916294 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:40.916266 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnkzr\" (UniqueName: \"kubernetes.io/projected/b0f87920-2518-4aa2-9db4-5959b7e71a54-kube-api-access-jnkzr\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:40.916294 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:40.916287 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-token\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:40.916361 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:40.916308 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-data\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:40.916421 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:40.916406 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-workload-certs\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.017334 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.017259 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-podinfo\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.017334 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.017295 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-workload-socket\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.017531 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.017361 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0f87920-2518-4aa2-9db4-5959b7e71a54-istiod-ca-cert\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.017531 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.017401 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-credential-socket\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.017531 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.017450 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnkzr\" (UniqueName: \"kubernetes.io/projected/b0f87920-2518-4aa2-9db4-5959b7e71a54-kube-api-access-jnkzr\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.017531 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.017478 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-token\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.017531 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.017505 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-data\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.017968 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.017557 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-workload-certs\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.017968 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.017590 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-envoy\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.017968 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.017768 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-workload-socket\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.018660 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.018635 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0f87920-2518-4aa2-9db4-5959b7e71a54-istiod-ca-cert\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.019071 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.019044 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-workload-certs\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.019189 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.019165 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-data\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.019472 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.019221 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-credential-socket\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.021065 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.021043 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-envoy\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.021182 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.021164 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-podinfo\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.040642 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.040624 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-token\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.040880 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.040862 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnkzr\" (UniqueName: \"kubernetes.io/projected/b0f87920-2518-4aa2-9db4-5959b7e71a54-kube-api-access-jnkzr\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.084876 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.084851 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:41.411840 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.411811 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5"] Oct 30 15:52:41.411949 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:52:41.411819 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f87920_2518_4aa2_9db4_5959b7e71a54.slice/crio-9ac08a20fceae24c02f7e04fb5c92914b165b6de70fe1394d520e6c30fa3e5b0 WatchSource:0}: Error finding container 9ac08a20fceae24c02f7e04fb5c92914b165b6de70fe1394d520e6c30fa3e5b0: Status 404 returned error can't find the container with id 9ac08a20fceae24c02f7e04fb5c92914b165b6de70fe1394d520e6c30fa3e5b0 Oct 30 15:52:41.414466 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.414434 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:52:41.414560 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.414497 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:52:41.414560 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:41.414527 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:52:42.247222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:42.240778 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" event={"ID":"b0f87920-2518-4aa2-9db4-5959b7e71a54","Type":"ContainerStarted","Data":"57f31ef6889321b22c87e27a59421508f1db555afdc72326ee9eb263ed1a8917"} Oct 30 15:52:42.248295 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:42.248265 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" event={"ID":"b0f87920-2518-4aa2-9db4-5959b7e71a54","Type":"ContainerStarted","Data":"9ac08a20fceae24c02f7e04fb5c92914b165b6de70fe1394d520e6c30fa3e5b0"} Oct 30 15:52:42.422124 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:42.422060 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" podStartSLOduration=2.422039443 podStartE2EDuration="2.422039443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:52:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:52:42.342701354 +0000 UTC m=+878.493598396" watchObservedRunningTime="2026-10-30 15:52:42.422039443 +0000 UTC m=+878.572936476" Oct 30 15:52:43.087313 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:43.087269 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:43.104466 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:43.104429 5013 patch_prober.go:29] interesting pod/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 15:52:43.104794 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:43.104762 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" podUID="b0f87920-2518-4aa2-9db4-5959b7e71a54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 15:52:43.765735 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:43.765703 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:44.095754 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:44.095673 5013 patch_prober.go:29] interesting pod/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 15:52:44.095754 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:44.095737 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" podUID="b0f87920-2518-4aa2-9db4-5959b7e71a54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 15:52:44.225287 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:44.225256 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:45.095797 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:45.095748 5013 patch_prober.go:29] interesting pod/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 15:52:45.096362 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:45.095821 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" podUID="b0f87920-2518-4aa2-9db4-5959b7e71a54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 15:52:46.012862 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:46.012828 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:46.090146 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:46.089673 5013 patch_prober.go:29] interesting pod/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 15:52:46.090146 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:46.089761 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" podUID="b0f87920-2518-4aa2-9db4-5959b7e71a54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 15:52:47.091677 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:47.091641 5013 patch_prober.go:29] interesting pod/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 15:52:47.092300 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:47.092258 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" podUID="b0f87920-2518-4aa2-9db4-5959b7e71a54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 15:52:48.091768 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:48.088604 5013 patch_prober.go:29] interesting pod/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 15:52:48.091768 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:48.088933 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" podUID="b0f87920-2518-4aa2-9db4-5959b7e71a54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 15:52:48.091768 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:48.089146 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:49.214970 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:49.214942 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:49.215870 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:49.215845 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:49.217828 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:49.217808 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:52:49.826668 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:49.826638 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:51.675704 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:51.675667 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:53.328903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:53.328871 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:54.952429 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:54.952398 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:55.116746 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:55.116679 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:56.702423 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.702391 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:52:56.754858 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.754818 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8"] Oct 30 15:52:56.760117 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.760093 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.775688 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.775643 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-c129d1-openshift-default-dockercfg-qvk6s\"" Oct 30 15:52:56.794366 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.794335 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8"] Oct 30 15:52:56.823069 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.823022 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnbn6\" (UniqueName: \"kubernetes.io/projected/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-kube-api-access-gnbn6\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.823283 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.823079 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.823283 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.823118 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.823283 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.823217 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.823283 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.823268 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.823455 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.823290 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.823455 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.823435 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.823560 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.823541 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.823627 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.823602 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.924963 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.924930 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnbn6\" (UniqueName: \"kubernetes.io/projected/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-kube-api-access-gnbn6\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.924963 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.924968 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.925248 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.924992 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.925248 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.925118 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.925248 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.925217 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.925424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.925255 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.925424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.925323 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.925424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.925387 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.925424 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.925392 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.925634 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.925429 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.925634 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.925465 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.925634 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.925588 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.925838 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.925815 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.925884 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.925827 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.927891 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.927863 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.928011 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.927993 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.948528 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.948496 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.948528 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:56.948518 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnbn6\" (UniqueName: \"kubernetes.io/projected/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-kube-api-access-gnbn6\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:56.983247 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:52:56.983127 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:52:57.095078 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:57.095038 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:57.443251 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:57.443171 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:52:57.443406 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:57.443301 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:52:57.443406 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:57.443349 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:52:57.444128 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:57.444108 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8"] Oct 30 15:52:58.395269 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:58.395232 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" event={"ID":"83bf7cb0-3eae-41b2-8ba0-4db7b4634071","Type":"ContainerStarted","Data":"788de54dd368efd887bbd708c8e08a82b2114bdf44f2cdc0e6d3912505afa07b"} Oct 30 15:52:58.395851 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:58.395827 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" event={"ID":"83bf7cb0-3eae-41b2-8ba0-4db7b4634071","Type":"ContainerStarted","Data":"8a861b7c6531ffc3ad89ef275ec9cc0ed402d2d42b934b3eeee96334dd74f4b6"} Oct 30 15:52:59.095412 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:59.095370 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:52:59.097382 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:59.097355 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Oct 30 15:52:59.097516 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:59.097404 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" podUID="83bf7cb0-3eae-41b2-8ba0-4db7b4634071" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Oct 30 15:52:59.146591 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:52:59.146555 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:00.095696 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:00.095652 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Oct 30 15:53:00.096113 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:00.095724 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" podUID="83bf7cb0-3eae-41b2-8ba0-4db7b4634071" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Oct 30 15:53:01.096221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:01.095678 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Oct 30 15:53:01.096221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:01.095749 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" podUID="83bf7cb0-3eae-41b2-8ba0-4db7b4634071" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Oct 30 15:53:01.096825 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:01.096697 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:02.095608 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:02.095568 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Oct 30 15:53:02.095781 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:02.095660 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" podUID="83bf7cb0-3eae-41b2-8ba0-4db7b4634071" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Oct 30 15:53:03.048366 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:03.048328 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:03.098648 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:03.098607 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Oct 30 15:53:03.098852 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:03.098686 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" podUID="83bf7cb0-3eae-41b2-8ba0-4db7b4634071" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Oct 30 15:53:04.086388 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:04.086356 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 15:53:04.086851 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:04.086640 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 15:53:04.095551 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:04.095521 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Oct 30 15:53:04.095707 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:04.095605 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" podUID="83bf7cb0-3eae-41b2-8ba0-4db7b4634071" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Oct 30 15:53:05.004450 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:05.004419 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:05.100051 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:05.100024 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:53:05.100468 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:05.100445 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:53:05.101137 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:05.101121 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:53:05.171783 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:05.171731 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" podStartSLOduration=9.171717526 podStartE2EDuration="9.171717526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:52:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:52:58.487226148 +0000 UTC m=+894.638123181" watchObservedRunningTime="2026-10-30 15:53:05.171717526 +0000 UTC m=+901.322614551" Oct 30 15:53:06.713051 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:06.713001 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:08.595778 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:08.595746 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:10.764363 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:10.764330 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:11.579249 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.579220 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl"] Oct 30 15:53:11.584428 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.584408 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.601440 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.601415 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-9e2b29-openshift-default-dockercfg-vfgdd\"" Oct 30 15:53:11.617667 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.617638 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl"] Oct 30 15:53:11.715528 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.715493 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.715700 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.715547 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.715700 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.715656 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.715844 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.715711 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.715844 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.715769 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.715844 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.715813 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.716022 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.715885 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9pq4\" (UniqueName: \"kubernetes.io/projected/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-kube-api-access-c9pq4\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.716022 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.715965 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.716022 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.716013 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.816946 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.816909 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9pq4\" (UniqueName: \"kubernetes.io/projected/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-kube-api-access-c9pq4\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.817425 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.816965 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.817425 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.816995 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.817425 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.817029 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.817425 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.817054 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.817425 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.817083 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.817425 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.817128 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.817425 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.817159 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.817425 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.817240 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.817887 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.817495 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.817887 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.817688 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.817887 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.817705 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.817887 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.817743 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.818172 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.818151 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.819643 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.819619 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.819798 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.819776 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.868020 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.867967 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9pq4\" (UniqueName: \"kubernetes.io/projected/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-kube-api-access-c9pq4\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.868202 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.868171 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:11.897886 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:11.897863 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:12.219708 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:12.219678 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl"] Oct 30 15:53:12.220023 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:53:12.219999 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef8ef827_bafc_4f76_96d3_4a5371e8bdaa.slice/crio-750be0493de0219648d8282577692916debbe4e6f4dc1fdbd2ec000c553890a4 WatchSource:0}: Error finding container 750be0493de0219648d8282577692916debbe4e6f4dc1fdbd2ec000c553890a4: Status 404 returned error can't find the container with id 750be0493de0219648d8282577692916debbe4e6f4dc1fdbd2ec000c553890a4 Oct 30 15:53:12.222511 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:12.222476 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:53:12.222606 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:12.222544 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:53:12.222606 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:12.222578 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:53:12.508957 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:12.508864 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" event={"ID":"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa","Type":"ContainerStarted","Data":"0955a0b0549ccf213519124395de6a34a9af8709494bf07c25403ebe67c48b00"} Oct 30 15:53:12.508957 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:12.508919 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" event={"ID":"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa","Type":"ContainerStarted","Data":"750be0493de0219648d8282577692916debbe4e6f4dc1fdbd2ec000c553890a4"} Oct 30 15:53:12.555482 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:12.555425 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" podStartSLOduration=1.5554137369999999 podStartE2EDuration="1.555413737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:53:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:53:12.553228584 +0000 UTC m=+908.704125616" watchObservedRunningTime="2026-10-30 15:53:12.555413737 +0000 UTC m=+908.706310768" Oct 30 15:53:12.870889 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:12.870780 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:12.898607 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:12.898560 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:13.900165 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:13.900127 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 15:53:13.900600 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:13.900223 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" podUID="ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 15:53:14.666579 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:14.666486 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:14.898583 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:14.898549 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 15:53:14.898748 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:14.898612 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" podUID="ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 15:53:15.898531 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:15.898483 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 15:53:15.898936 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:15.898560 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" podUID="ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 15:53:16.497651 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:16.497616 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:16.898595 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:16.898560 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 15:53:16.899130 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:16.898628 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" podUID="ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 15:53:17.898800 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:17.898759 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 15:53:17.899182 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:17.898837 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" podUID="ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 15:53:18.410753 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:18.410697 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:19.147796 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:19.147761 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:19.148727 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:19.148671 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:19.149840 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:19.149819 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:53:20.494484 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:20.494450 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:22.258442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:22.258411 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:24.076678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:24.076649 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:26.132335 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:26.132280 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded" Oct 30 15:53:26.683221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:26.681588 5013 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": context deadline exceeded" Oct 30 15:53:26.700814 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:26.700775 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:53:27.262220 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:53:27.260628 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:53:27.508634 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:27.508600 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:28.634876 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:28.634844 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:29.471519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:29.471488 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:29.636271 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:29.636238 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:31.285726 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:31.285696 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:32.994278 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:32.994170 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:34.733072 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:34.733032 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:34.897355 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:34.897321 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:36.586605 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:36.586571 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:38.261017 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:38.260979 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:39.968085 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:39.968043 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:41.917450 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:41.917414 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:44.302105 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:44.301816 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:46.075146 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:46.075106 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:47.376056 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.376024 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf"] Oct 30 15:53:47.381556 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.381535 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.404493 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.404449 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf"] Oct 30 15:53:47.409428 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.409403 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-b63c3c\"/\"kube-root-ca.crt\"" Oct 30 15:53:47.412106 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.412081 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-b63c3c\"/\"openshift-service-ca.crt\"" Oct 30 15:53:47.413094 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.413071 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b63c3c\"/\"default-dockercfg-wksm9\"" Oct 30 15:53:47.413094 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.413089 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b63c3c\"/\"distinct-w0-b63c3c-kserve-self-signed-certs\"" Oct 30 15:53:47.440841 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.440790 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-tmp-dir\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.441253 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.441225 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-home\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.441374 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.441322 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-model-cache\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.441374 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.441349 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmg7m\" (UniqueName: \"kubernetes.io/projected/017d73bc-9fcc-4c13-bfdd-af21ac301a65-kube-api-access-dmg7m\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.441474 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.441415 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/017d73bc-9fcc-4c13-bfdd-af21ac301a65-tls-certs\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.441474 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.441459 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-dshm\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.504493 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.504457 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:47.542831 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.542804 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-tmp-dir\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.543007 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.542873 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-home\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.543007 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.542920 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-model-cache\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.543007 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.542945 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmg7m\" (UniqueName: \"kubernetes.io/projected/017d73bc-9fcc-4c13-bfdd-af21ac301a65-kube-api-access-dmg7m\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.543007 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.542996 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/017d73bc-9fcc-4c13-bfdd-af21ac301a65-tls-certs\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.543264 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.543032 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-dshm\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.543264 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.543251 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-tmp-dir\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.543370 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.543310 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-home\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.543370 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.543356 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-model-cache\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.546537 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.546512 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/017d73bc-9fcc-4c13-bfdd-af21ac301a65-tls-certs\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.548392 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.548369 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-dshm\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.567044 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.567013 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmg7m\" (UniqueName: \"kubernetes.io/projected/017d73bc-9fcc-4c13-bfdd-af21ac301a65-kube-api-access-dmg7m\") pod \"distinct-w0-b63c3c-kserve-595dffc77c-nxrnf\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.699446 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.699416 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:47.795340 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:47.795306 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:48.014343 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:48.014317 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf"] Oct 30 15:53:48.015690 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:53:48.015664 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod017d73bc_9fcc_4c13_bfdd_af21ac301a65.slice/crio-48a3fe27491c02f7f5c2865794555e1c88159eeabd055f4b263e0c405a2fa35a WatchSource:0}: Error finding container 48a3fe27491c02f7f5c2865794555e1c88159eeabd055f4b263e0c405a2fa35a: Status 404 returned error can't find the container with id 48a3fe27491c02f7f5c2865794555e1c88159eeabd055f4b263e0c405a2fa35a Oct 30 15:53:48.856899 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:48.856865 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" event={"ID":"017d73bc-9fcc-4c13-bfdd-af21ac301a65","Type":"ContainerStarted","Data":"65f0d73189509e41dcac2d0992620d304058ad9f5f77db7d289f873a2614e3b2"} Oct 30 15:53:48.857301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:48.856907 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" event={"ID":"017d73bc-9fcc-4c13-bfdd-af21ac301a65","Type":"ContainerStarted","Data":"48a3fe27491c02f7f5c2865794555e1c88159eeabd055f4b263e0c405a2fa35a"} Oct 30 15:53:48.897246 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:48.897169 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podStartSLOduration=1.897156759 podStartE2EDuration="1.897156759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:53:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:53:48.896578563 +0000 UTC m=+945.047475594" watchObservedRunningTime="2026-10-30 15:53:48.897156759 +0000 UTC m=+945.048053789" Oct 30 15:53:49.516573 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:49.516542 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:49.700472 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:49.700445 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:51.283559 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:51.283526 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:52.762147 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.762111 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn"] Oct 30 15:53:52.768055 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.768005 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.797893 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.797861 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn"] Oct 30 15:53:52.802461 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.802436 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.802602 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.802475 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.802602 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.802501 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.802602 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.802515 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.802602 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.802580 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.802602 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.802595 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.802870 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.802615 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.802870 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.802647 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpntv\" (UniqueName: \"kubernetes.io/projected/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-kube-api-access-gpntv\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.802870 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.802665 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.904268 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.904234 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.904268 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.904265 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.904502 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.904334 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.904502 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.904352 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.904502 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.904386 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.904502 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.904434 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpntv\" (UniqueName: \"kubernetes.io/projected/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-kube-api-access-gpntv\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.904502 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.904465 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.904781 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.904591 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.904781 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.904637 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.904781 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.904767 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.904944 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.904882 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.905007 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.904941 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.905109 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.905088 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.905163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.905117 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.907027 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.907005 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.907290 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.907271 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.926998 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.926967 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:52.927141 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:52.927124 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpntv\" (UniqueName: \"kubernetes.io/projected/3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e-kube-api-access-gpntv\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-bl2mn\" (UID: \"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:53.084461 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:53.084396 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:53.178240 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:53.174101 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:53.401465 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:53:53.401429 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a73d3e0_afac_4a0a_b83c_4d2baaa9db1e.slice/crio-6dd1c0bd7f3c7626161a09a6ed409fcec396e51527836351e503bc9f3f8e66f6 WatchSource:0}: Error finding container 6dd1c0bd7f3c7626161a09a6ed409fcec396e51527836351e503bc9f3f8e66f6: Status 404 returned error can't find the container with id 6dd1c0bd7f3c7626161a09a6ed409fcec396e51527836351e503bc9f3f8e66f6 Oct 30 15:53:53.402161 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:53.402136 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn"] Oct 30 15:53:53.404078 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:53.404047 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:53:53.404173 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:53.404110 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:53:53.404173 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:53.404147 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:53:53.891211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:53.891170 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" event={"ID":"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e","Type":"ContainerStarted","Data":"de34991b67902980f7ca93c9254bfc34cd1ae4405195b8d6b1e3a586807163e9"} Oct 30 15:53:53.891597 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:53.891269 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" event={"ID":"3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e","Type":"ContainerStarted","Data":"6dd1c0bd7f3c7626161a09a6ed409fcec396e51527836351e503bc9f3f8e66f6"} Oct 30 15:53:53.939394 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:53.939347 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" podStartSLOduration=1.9393332189999999 podStartE2EDuration="1.939333219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:53:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:53:53.938699586 +0000 UTC m=+950.089596618" watchObservedRunningTime="2026-10-30 15:53:53.939333219 +0000 UTC m=+950.090230250" Oct 30 15:53:54.084624 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:54.084595 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:53:54.086480 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:54.086455 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 15:53:54.086600 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:54.086503 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" podUID="3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 15:53:55.085630 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:55.085586 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 15:53:55.086148 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:55.085648 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" podUID="3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 15:53:55.090705 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:55.090672 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:55.261488 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:55.261458 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:55.812462 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:55.812428 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:56.088672 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:56.088582 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 15:53:56.089248 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:56.089186 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" podUID="3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 15:53:56.996410 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:56.995991 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:57.022718 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:53:57.022658 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:53:57.095278 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.091538 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 15:53:57.095278 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.091607 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" podUID="3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 15:53:57.235286 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.235250 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr"] Oct 30 15:53:57.239737 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.239716 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.270044 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.270019 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-b24e7b-openshift-default-dockercfg-fsj9p\"" Oct 30 15:53:57.301670 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.301633 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr"] Oct 30 15:53:57.360450 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.360412 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.360608 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.360490 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.360608 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.360596 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.360680 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.360637 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.360680 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.360656 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.360759 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.360741 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.360794 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.360774 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.360850 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.360837 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.360894 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.360883 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x9f4\" (UniqueName: \"kubernetes.io/projected/f0b22fc2-cafd-4d44-ab78-d1020db061fa-kube-api-access-8x9f4\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.462570 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.462538 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.462728 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.462593 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x9f4\" (UniqueName: \"kubernetes.io/projected/f0b22fc2-cafd-4d44-ab78-d1020db061fa-kube-api-access-8x9f4\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.462728 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.462625 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.462728 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.462673 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.462728 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.462722 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.462902 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.462756 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.462902 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.462780 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.462902 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.462818 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.462902 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.462844 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.463113 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.463080 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.463173 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.463146 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.463289 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.463254 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.463386 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.463366 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.463832 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.463809 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.465067 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.465049 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.465325 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.465308 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.496878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.496850 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.497825 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.497756 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x9f4\" (UniqueName: \"kubernetes.io/projected/f0b22fc2-cafd-4d44-ab78-d1020db061fa-kube-api-access-8x9f4\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.553743 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.553716 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:57.701219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.701159 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:57.717145 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.717112 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:57.912697 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.912668 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr"] Oct 30 15:53:57.914185 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:53:57.914159 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0b22fc2_cafd_4d44_ab78_d1020db061fa.slice/crio-db2ea4fd9dd37d2b2157b506be6bd9335385f5fa8164a80de0bcaca2d74799f7 WatchSource:0}: Error finding container db2ea4fd9dd37d2b2157b506be6bd9335385f5fa8164a80de0bcaca2d74799f7: Status 404 returned error can't find the container with id db2ea4fd9dd37d2b2157b506be6bd9335385f5fa8164a80de0bcaca2d74799f7 Oct 30 15:53:57.916926 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.916883 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:53:57.917036 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.916957 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:53:57.917036 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.916989 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:53:57.946164 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.946135 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" event={"ID":"f0b22fc2-cafd-4d44-ab78-d1020db061fa","Type":"ContainerStarted","Data":"db2ea4fd9dd37d2b2157b506be6bd9335385f5fa8164a80de0bcaca2d74799f7"} Oct 30 15:53:57.958992 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:57.958969 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:53:58.088011 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:58.087535 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 15:53:58.088011 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:58.087594 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" podUID="3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 15:53:58.998223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:58.998166 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" event={"ID":"f0b22fc2-cafd-4d44-ab78-d1020db061fa","Type":"ContainerStarted","Data":"f3e72b8621810f24e8b164e56431e1530ab55df8ad5a2ee8cd5865d2b771bbe7"} Oct 30 15:53:59.089599 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:59.089562 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 15:53:59.093295 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:59.093256 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" podUID="3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 15:53:59.117602 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:59.117539 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" podStartSLOduration=2.117516484 podStartE2EDuration="2.117516484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:53:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:53:59.103229898 +0000 UTC m=+955.254126939" watchObservedRunningTime="2026-10-30 15:53:59.117516484 +0000 UTC m=+955.268413516" Oct 30 15:53:59.559644 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:59.559604 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:53:59.561721 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:59.561688 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 15:53:59.561980 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:59.561936 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" podUID="f0b22fc2-cafd-4d44-ab78-d1020db061fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 15:53:59.891251 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:59.891133 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:53:59.928294 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:59.928243 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 15:53:59.952248 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:53:59.951272 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:54:00.132223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:00.111919 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 15:54:00.132223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:00.112006 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" podUID="3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 15:54:00.554370 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:00.554336 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 15:54:00.554558 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:00.554398 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" podUID="f0b22fc2-cafd-4d44-ab78-d1020db061fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 15:54:01.087222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:01.085541 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 15:54:01.087222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:01.085614 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" podUID="3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 15:54:01.554276 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:01.554240 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 15:54:01.554744 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:01.554300 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" podUID="f0b22fc2-cafd-4d44-ab78-d1020db061fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 15:54:01.767644 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:01.767546 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:02.097389 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:02.097351 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 15:54:02.097657 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:02.097631 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" podUID="3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 15:54:02.555068 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:02.555030 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 15:54:02.555591 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:02.555082 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" podUID="f0b22fc2-cafd-4d44-ab78-d1020db061fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 15:54:03.149903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:03.149872 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:54:03.150664 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:03.150636 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:54:03.151493 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:03.151473 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" Oct 30 15:54:03.554967 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:03.554896 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 15:54:03.554967 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:03.554949 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" podUID="f0b22fc2-cafd-4d44-ab78-d1020db061fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 15:54:03.774221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:03.770500 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:04.554299 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:04.554261 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 15:54:04.554460 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:04.554328 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" podUID="f0b22fc2-cafd-4d44-ab78-d1020db061fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 15:54:05.681106 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:05.681067 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:06.223442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:06.223410 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:54:06.224114 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:06.224091 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:54:06.225341 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:06.225314 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:54:07.380078 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:07.380035 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:09.185832 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:09.185794 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:10.463765 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:10.463726 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:10.994505 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:10.994476 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:12.829939 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:12.829907 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:14.553576 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:14.553529 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:16.265678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:16.265643 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:18.085544 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:18.085512 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:19.830445 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:19.830414 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:20.884755 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:20.884717 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:21.588904 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:21.588865 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:23.305504 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:23.305470 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:25.040921 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:25.040890 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:26.950435 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:26.950395 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:26.989600 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:54:26.989565 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:54:28.680927 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:28.680888 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:30.490890 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:30.490854 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:32.281322 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:32.281285 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:34.055665 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:34.055638 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:35.786316 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:35.786282 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:37.798550 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:37.798506 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:38.385019 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.384988 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t"] Oct 30 15:54:38.390982 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.390960 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.410031 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.410000 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t"] Oct 30 15:54:38.415261 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.415233 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b24e7b\"/\"kube-root-ca.crt\"" Oct 30 15:54:38.415408 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.415318 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b24e7b\"/\"default-dockercfg-h2s6c\"" Oct 30 15:54:38.418381 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.418358 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b24e7b\"/\"openshift-service-ca.crt\"" Oct 30 15:54:38.418549 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.418433 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b24e7b\"/\"distinct-2-w4-b24e7b-kserve-self-signed-certs\"" Oct 30 15:54:38.486073 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.486038 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9d63587-9839-45d5-8bbe-e5208b3446b6-tls-certs\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.486294 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.486277 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-model-cache\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.486371 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.486347 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-dshm\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.486428 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.486376 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-tmp-dir\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.486428 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.486418 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-home\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.486540 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.486459 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptgn8\" (UniqueName: \"kubernetes.io/projected/f9d63587-9839-45d5-8bbe-e5208b3446b6-kube-api-access-ptgn8\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.508766 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.508735 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:38.589218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.587669 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-model-cache\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.589218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.587746 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-dshm\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.589218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.587775 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-tmp-dir\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.589218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.587809 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-home\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.589218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.587836 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptgn8\" (UniqueName: \"kubernetes.io/projected/f9d63587-9839-45d5-8bbe-e5208b3446b6-kube-api-access-ptgn8\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.589218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.587933 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9d63587-9839-45d5-8bbe-e5208b3446b6-tls-certs\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.589218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.588114 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-model-cache\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.589218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.588423 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-tmp-dir\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.589218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.588983 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-home\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.591950 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.591923 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9d63587-9839-45d5-8bbe-e5208b3446b6-tls-certs\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.617432 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.617366 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptgn8\" (UniqueName: \"kubernetes.io/projected/f9d63587-9839-45d5-8bbe-e5208b3446b6-kube-api-access-ptgn8\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.619055 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.618997 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-dshm\") pod \"distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:38.708834 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:38.708510 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:39.038247 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:39.038218 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t"] Oct 30 15:54:39.038653 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:54:39.038597 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9d63587_9839_45d5_8bbe_e5208b3446b6.slice/crio-132eab97d0840e78c794cd0d94aa76bd59699f5acdc6f7d044dcf3c3e16e828f WatchSource:0}: Error finding container 132eab97d0840e78c794cd0d94aa76bd59699f5acdc6f7d044dcf3c3e16e828f: Status 404 returned error can't find the container with id 132eab97d0840e78c794cd0d94aa76bd59699f5acdc6f7d044dcf3c3e16e828f Oct 30 15:54:39.278661 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:39.278621 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" event={"ID":"f9d63587-9839-45d5-8bbe-e5208b3446b6","Type":"ContainerStarted","Data":"2e5d4c345159d98344800fd31cc506a69c9b6267e4e921d75b1f9ae659d489d3"} Oct 30 15:54:39.278661 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:39.278663 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" event={"ID":"f9d63587-9839-45d5-8bbe-e5208b3446b6","Type":"ContainerStarted","Data":"132eab97d0840e78c794cd0d94aa76bd59699f5acdc6f7d044dcf3c3e16e828f"} Oct 30 15:54:39.319389 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:39.319309 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podStartSLOduration=1.319294481 podStartE2EDuration="1.319294481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:54:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:54:39.318684914 +0000 UTC m=+995.469581945" watchObservedRunningTime="2026-10-30 15:54:39.319294481 +0000 UTC m=+995.470191515" Oct 30 15:54:39.709705 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:39.709676 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:39.745255 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:39.743756 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:41.471397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:41.471362 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:42.277761 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.277729 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996"] Oct 30 15:54:42.282018 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.282001 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.316391 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.316363 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996"] Oct 30 15:54:42.320748 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.320722 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c"] Oct 30 15:54:42.330525 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.330503 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.333836 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.333806 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.333937 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.333878 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.333990 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.333949 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.333990 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.333975 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.334142 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.334122 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vn6r\" (UniqueName: \"kubernetes.io/projected/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-kube-api-access-5vn6r\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.334236 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.334221 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.334300 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.334248 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.334300 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.334279 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.334404 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.334336 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.366233 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.363991 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c"] Oct 30 15:54:42.435969 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.435940 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.436106 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.435992 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.436106 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436068 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vn6r\" (UniqueName: \"kubernetes.io/projected/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-kube-api-access-5vn6r\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.436232 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436106 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.436232 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436166 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.436318 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436230 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.436318 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436269 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.436318 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436310 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9skdx\" (UniqueName: \"kubernetes.io/projected/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-kube-api-access-9skdx\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.436529 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436341 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.436529 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436423 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.436529 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436494 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.436676 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436531 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.436676 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436567 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.436676 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436667 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.436783 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436716 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.436783 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436752 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.436897 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436868 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.436952 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436931 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.437003 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436967 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.437003 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.436995 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.437102 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.437026 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.437323 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.437298 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.437407 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.437349 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.438678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.438657 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.439242 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.439222 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.460908 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.460882 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vn6r\" (UniqueName: \"kubernetes.io/projected/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-kube-api-access-5vn6r\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.461815 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.461796 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.538445 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.538384 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.538827 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.538457 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.538827 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.538554 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9skdx\" (UniqueName: \"kubernetes.io/projected/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-kube-api-access-9skdx\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.538827 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.538589 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.538827 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.538632 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.538827 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.538662 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.538827 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.538720 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.538827 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.538750 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.539178 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.538849 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.539308 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.539291 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.539484 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.539462 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.539558 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.539500 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.539611 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.539576 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.539611 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.539600 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.541236 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.541216 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.541654 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.541636 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.560995 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.560972 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.561158 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.561142 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9skdx\" (UniqueName: \"kubernetes.io/projected/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-kube-api-access-9skdx\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.600041 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.600020 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:42.648314 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.648282 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:42.932139 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:54:42.932112 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb01099_f910_4f21_a0b6_ff5dbc33bf60.slice/crio-bb16090180f849648455bca0b9708221cd670a7647dc3810c8fca1bf0327173e WatchSource:0}: Error finding container bb16090180f849648455bca0b9708221cd670a7647dc3810c8fca1bf0327173e: Status 404 returned error can't find the container with id bb16090180f849648455bca0b9708221cd670a7647dc3810c8fca1bf0327173e Oct 30 15:54:42.932407 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.932385 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996"] Oct 30 15:54:42.934625 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.934590 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:54:42.934727 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.934663 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:54:42.934727 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.934704 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:54:42.996205 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:42.996160 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c"] Oct 30 15:54:42.997061 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:54:42.997039 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46ca9dff_35b7_40c0_a2c1_ca1b0b77ff92.slice/crio-577cc67fe9df1a1221022867a2666448d7f0690cc3944fa63d7c0b3c00a69875 WatchSource:0}: Error finding container 577cc67fe9df1a1221022867a2666448d7f0690cc3944fa63d7c0b3c00a69875: Status 404 returned error can't find the container with id 577cc67fe9df1a1221022867a2666448d7f0690cc3944fa63d7c0b3c00a69875 Oct 30 15:54:43.000235 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:43.000180 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:54:43.000322 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:43.000278 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:54:43.000362 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:43.000319 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:54:43.183292 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:43.183088 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:43.306386 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:43.306274 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" event={"ID":"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92","Type":"ContainerStarted","Data":"e497f068bb92fdfcb5f000d2e2e08ed0a37856789aa3ec3c702092def1545b21"} Oct 30 15:54:43.306386 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:43.306343 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" event={"ID":"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92","Type":"ContainerStarted","Data":"577cc67fe9df1a1221022867a2666448d7f0690cc3944fa63d7c0b3c00a69875"} Oct 30 15:54:43.309954 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:43.309923 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" event={"ID":"0cb01099-f910-4f21-a0b6-ff5dbc33bf60","Type":"ContainerStarted","Data":"90543c34d4a9022aebfb8229aa56e3712f6323b98e0b34ad6e555068f3887c10"} Oct 30 15:54:43.310087 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:43.309982 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" event={"ID":"0cb01099-f910-4f21-a0b6-ff5dbc33bf60","Type":"ContainerStarted","Data":"bb16090180f849648455bca0b9708221cd670a7647dc3810c8fca1bf0327173e"} Oct 30 15:54:43.351874 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:43.351819 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" podStartSLOduration=1.35180099 podStartE2EDuration="1.35180099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:54:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:54:43.350599174 +0000 UTC m=+999.501496216" watchObservedRunningTime="2026-10-30 15:54:43.35180099 +0000 UTC m=+999.502698028" Oct 30 15:54:43.398122 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:43.398060 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" podStartSLOduration=1.398041381 podStartE2EDuration="1.398041381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:54:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:54:43.39463859 +0000 UTC m=+999.545535624" watchObservedRunningTime="2026-10-30 15:54:43.398041381 +0000 UTC m=+999.548938413" Oct 30 15:54:43.600419 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:43.600312 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:43.649400 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:43.649363 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:44.612218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:44.611435 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 30 15:54:44.612218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:44.611502 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" podUID="0cb01099-f910-4f21-a0b6-ff5dbc33bf60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 30 15:54:44.652613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:44.652582 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 15:54:44.652808 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:44.652628 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" podUID="46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 15:54:45.097222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:45.094773 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:45.601220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:45.600539 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 30 15:54:45.601220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:45.600594 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" podUID="0cb01099-f910-4f21-a0b6-ff5dbc33bf60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 30 15:54:45.649212 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:45.649070 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 15:54:45.649212 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:45.649131 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" podUID="46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 15:54:46.614219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:46.614147 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 30 15:54:46.614444 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:46.614229 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" podUID="0cb01099-f910-4f21-a0b6-ff5dbc33bf60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 30 15:54:46.648764 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:46.648729 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 15:54:46.648913 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:46.648789 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" podUID="46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 15:54:47.362083 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:47.362053 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:47.601220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:47.600580 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 30 15:54:47.601220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:47.600639 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" podUID="0cb01099-f910-4f21-a0b6-ff5dbc33bf60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 30 15:54:47.649064 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:47.649029 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 15:54:47.649264 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:47.649091 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" podUID="46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 15:54:48.619222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:48.614639 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 30 15:54:48.619222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:48.614706 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" podUID="0cb01099-f910-4f21-a0b6-ff5dbc33bf60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 30 15:54:48.648720 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:48.648691 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 15:54:48.648863 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:48.648740 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" podUID="46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 15:54:48.715220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:48.714576 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:48.733392 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:48.733366 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:49.364559 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:49.364518 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:49.377644 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:49.377615 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:54:49.600631 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:49.600596 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 30 15:54:49.600800 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:49.600669 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" podUID="0cb01099-f910-4f21-a0b6-ff5dbc33bf60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 30 15:54:49.649081 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:49.649048 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 15:54:49.649475 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:49.649109 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" podUID="46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 15:54:50.604731 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:50.604694 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:50.605140 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:50.605114 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:50.605932 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:50.605911 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:54:50.668923 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:50.668877 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:50.669664 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:50.669632 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:50.670556 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:50.670538 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:54:51.244550 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:51.244496 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:53.471459 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:53.471411 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:55.214448 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:55.214406 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:55.381837 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:55.381800 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:56.978324 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:54:56.978281 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:54:57.015546 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:57.015512 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:54:58.763557 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:54:58.763516 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:00.584494 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:00.584457 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:02.290822 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:02.290765 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:04.038387 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:04.038356 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:05.789659 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:05.789623 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:07.494402 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:07.494370 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:09.465221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:09.451487 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:09.621860 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:09.621825 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:09.799719 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:09.799632 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:10.694220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:10.693891 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:10.924224 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:10.922329 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:10.962279 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:10.955446 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:11.025218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:11.019290 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:12.023105 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.023065 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:12.497075 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.497037 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz"] Oct 30 15:55:12.503309 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.503283 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.536863 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.536834 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz"] Oct 30 15:55:12.661751 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.661714 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.661932 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.661847 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.661932 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.661905 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.662029 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.662007 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.662067 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.662039 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.662112 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.662068 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4t67\" (UniqueName: \"kubernetes.io/projected/c0fda42f-3851-4542-a220-b9591e30d2a7-kube-api-access-z4t67\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.662247 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.662227 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.662315 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.662272 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.662371 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.662311 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0fda42f-3851-4542-a220-b9591e30d2a7-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.763600 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.763503 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.763600 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.763568 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4t67\" (UniqueName: \"kubernetes.io/projected/c0fda42f-3851-4542-a220-b9591e30d2a7-kube-api-access-z4t67\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.763831 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.763632 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.763831 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.763679 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.763831 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.763702 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0fda42f-3851-4542-a220-b9591e30d2a7-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.763831 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.763744 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.764048 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.763845 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.764048 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.763888 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.764048 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.763987 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.764243 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.764143 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.764484 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.764329 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.764862 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.764656 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.765489 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.765279 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.765489 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.765442 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0fda42f-3851-4542-a220-b9591e30d2a7-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.766946 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.766909 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.767318 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.767299 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.787268 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.787183 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.788117 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.788096 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4t67\" (UniqueName: \"kubernetes.io/projected/c0fda42f-3851-4542-a220-b9591e30d2a7-kube-api-access-z4t67\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:12.819185 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:12.819160 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:13.139973 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:13.139938 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz"] Oct 30 15:55:13.142164 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:55:13.142140 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0fda42f_3851_4542_a220_b9591e30d2a7.slice/crio-3ab09b82996288641c57070d17e75dc887987fbecbb82a4b9318f82cd982c325 WatchSource:0}: Error finding container 3ab09b82996288641c57070d17e75dc887987fbecbb82a4b9318f82cd982c325: Status 404 returned error can't find the container with id 3ab09b82996288641c57070d17e75dc887987fbecbb82a4b9318f82cd982c325 Oct 30 15:55:13.145125 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:13.145088 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:55:13.145266 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:13.145155 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:55:13.145327 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:13.145186 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:55:13.583677 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:13.583578 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" event={"ID":"c0fda42f-3851-4542-a220-b9591e30d2a7","Type":"ContainerStarted","Data":"2b3bd4ec04774e632314ac6f294d13dcad563bdaa43d58044f7a03f24d235dd4"} Oct 30 15:55:13.583677 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:13.583645 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" event={"ID":"c0fda42f-3851-4542-a220-b9591e30d2a7","Type":"ContainerStarted","Data":"3ab09b82996288641c57070d17e75dc887987fbecbb82a4b9318f82cd982c325"} Oct 30 15:55:13.645562 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:13.645507 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" podStartSLOduration=1.645492941 podStartE2EDuration="1.645492941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:55:13.643958984 +0000 UTC m=+1029.794856016" watchObservedRunningTime="2026-10-30 15:55:13.645492941 +0000 UTC m=+1029.796390036" Oct 30 15:55:13.819568 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:13.819492 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:14.118973 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:14.118938 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:14.821394 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:14.821362 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 15:55:14.821845 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:14.821420 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" podUID="c0fda42f-3851-4542-a220-b9591e30d2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 15:55:15.820224 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:15.820176 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 15:55:15.820434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:15.820254 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" podUID="c0fda42f-3851-4542-a220-b9591e30d2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 15:55:16.116586 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:16.116467 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:16.819706 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:16.819673 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 15:55:16.819958 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:16.819927 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" podUID="c0fda42f-3851-4542-a220-b9591e30d2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 15:55:17.819701 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:17.819660 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 15:55:17.820160 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:17.819722 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" podUID="c0fda42f-3851-4542-a220-b9591e30d2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 15:55:17.892323 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:17.892257 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:18.820152 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:18.820121 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 15:55:18.820565 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:18.820244 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" podUID="c0fda42f-3851-4542-a220-b9591e30d2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 15:55:19.735592 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:19.735555 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:19.819575 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:19.819541 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 15:55:19.819771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:19.819595 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" podUID="c0fda42f-3851-4542-a220-b9591e30d2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 15:55:20.823423 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:20.823395 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:20.823876 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:20.823857 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:20.824689 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:20.824669 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:55:21.568628 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:21.568600 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:23.366506 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:23.366473 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:25.337177 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:25.337140 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:26.989796 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:55:26.989747 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:55:27.496331 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.496298 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:27.546634 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.546594 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq"] Oct 30 15:55:27.551785 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.551761 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.574134 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.574108 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq"] Oct 30 15:55:27.681094 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.681055 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.681273 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.681151 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.681273 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.681177 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.681388 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.681275 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zdww\" (UniqueName: \"kubernetes.io/projected/1b325488-f2d5-49c8-9c76-d70554d9f0fe-kube-api-access-4zdww\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.681460 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.681436 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.681518 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.681480 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.681576 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.681554 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.681677 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.681662 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.681717 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.681688 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.782731 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.782664 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.782731 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.782702 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.782944 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.782752 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.782944 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.782794 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.782944 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.782907 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.782944 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.782934 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zdww\" (UniqueName: \"kubernetes.io/projected/1b325488-f2d5-49c8-9c76-d70554d9f0fe-kube-api-access-4zdww\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.783209 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.783040 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.783209 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.783072 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.783209 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.783102 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.783209 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.783164 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.783506 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.783485 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.783870 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.783826 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.783870 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.783854 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.784363 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.784327 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.785863 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.785830 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.785974 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.785954 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.805856 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.805822 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zdww\" (UniqueName: \"kubernetes.io/projected/1b325488-f2d5-49c8-9c76-d70554d9f0fe-kube-api-access-4zdww\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.810597 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.810575 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:27.873827 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:27.873798 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:28.212400 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:55:28.212371 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b325488_f2d5_49c8_9c76_d70554d9f0fe.slice/crio-809644c0e2f49271e8d9aec587c31fd3028fb2995908d51ebae3d6b0270cd85a WatchSource:0}: Error finding container 809644c0e2f49271e8d9aec587c31fd3028fb2995908d51ebae3d6b0270cd85a: Status 404 returned error can't find the container with id 809644c0e2f49271e8d9aec587c31fd3028fb2995908d51ebae3d6b0270cd85a Oct 30 15:55:28.213295 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:28.213271 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq"] Oct 30 15:55:28.214829 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:28.214794 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:55:28.214925 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:28.214859 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:55:28.214925 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:28.214888 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:55:28.727183 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:28.727149 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" event={"ID":"1b325488-f2d5-49c8-9c76-d70554d9f0fe","Type":"ContainerStarted","Data":"35e78a306d5627a5a97aac734bf36666209444de7ac21fe33d7538f9a9a6c3cd"} Oct 30 15:55:28.727397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:28.727250 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" event={"ID":"1b325488-f2d5-49c8-9c76-d70554d9f0fe","Type":"ContainerStarted","Data":"809644c0e2f49271e8d9aec587c31fd3028fb2995908d51ebae3d6b0270cd85a"} Oct 30 15:55:28.772509 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:28.772445 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" podStartSLOduration=1.7724309200000001 podStartE2EDuration="1.77243092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:55:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:55:28.771381075 +0000 UTC m=+1044.922278108" watchObservedRunningTime="2026-10-30 15:55:28.77243092 +0000 UTC m=+1044.923327951" Oct 30 15:55:28.875247 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:28.874918 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:29.371328 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:29.371240 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:29.898345 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:29.898307 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 15:55:29.898531 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:29.898375 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" podUID="1b325488-f2d5-49c8-9c76-d70554d9f0fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 15:55:30.882583 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:30.882546 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 15:55:30.883186 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:30.883154 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" podUID="1b325488-f2d5-49c8-9c76-d70554d9f0fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 15:55:31.148802 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:31.148709 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:31.895221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:31.891576 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 15:55:31.895221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:31.891638 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" podUID="1b325488-f2d5-49c8-9c76-d70554d9f0fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 15:55:32.896640 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:32.896603 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 15:55:32.897254 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:32.897218 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" podUID="1b325488-f2d5-49c8-9c76-d70554d9f0fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 15:55:32.992255 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:32.992220 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:33.880222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:33.878564 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 15:55:33.880222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:33.878631 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" podUID="1b325488-f2d5-49c8-9c76-d70554d9f0fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 15:55:34.865482 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:34.865430 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:34.874289 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:34.874255 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 15:55:34.874455 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:34.874326 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" podUID="1b325488-f2d5-49c8-9c76-d70554d9f0fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 15:55:35.879124 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:35.879095 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:35.879628 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:35.879527 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:35.880375 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:35.880351 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:55:36.518549 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:36.518517 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:38.367565 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:38.367529 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:40.100937 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:40.100906 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:41.943704 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:41.943667 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:44.356219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:44.355329 5013 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:44.366321 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:44.366276 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:44.695391 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:44.695341 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:44.886359 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:44.886306 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:44.979388 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:44.979285 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:45.012651 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:45.012607 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:45.129323 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:45.129281 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:45.375372 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:45.375279 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:45.376673 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:45.376650 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:55:45.681217 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:45.656340 5013 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:45.868853 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:45.837286 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded" Oct 30 15:55:45.868853 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:45.837487 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:55:45.887402 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:45.887368 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded" Oct 30 15:55:45.888071 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:45.888052 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:55:45.991275 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:45.990257 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:45.991275 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:45.990919 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:55:46.009213 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:45.995804 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:55:46.063884 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:46.063859 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:55:46.075457 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:46.075345 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:46.376237 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:46.376133 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:55:46.540153 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:46.540042 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:55:46.646170 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:46.646141 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:55:47.830174 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:47.830143 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:49.620023 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:49.619986 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:51.257913 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:51.257875 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:53.092587 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:53.092532 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:54.871450 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:54.871415 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:55.339407 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:55.339372 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:56.573761 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:56.573667 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:55:56.977558 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:55:56.977519 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:55:58.250104 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:55:58.250068 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:00.138219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:00.137905 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:00.502667 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:00.502584 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:01.040356 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:01.040311 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:01.070327 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:01.070278 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:01.134330 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:01.134282 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:01.544489 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:01.543290 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:01.649360 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:01.649313 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:01.959219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:01.958288 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:02.070131 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:02.070057 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded" Oct 30 15:56:02.071174 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:02.071142 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:02.138785 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:02.138755 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:56:02.152221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:02.139048 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:56:02.959287 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:02.959223 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:02.960464 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:02.960436 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:56:03.079131 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:03.074470 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:03.137707 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:03.136982 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:03.161792 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:03.145183 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:56:03.244745 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:03.244667 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:56:03.309882 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:03.309845 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:56:03.453423 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:03.453358 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:05.542665 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:05.542636 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:07.200486 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:07.200438 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:08.847286 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:08.847183 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:10.691794 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:10.691756 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:12.412744 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:12.412708 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:14.106025 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:14.105990 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:16.135220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:16.133285 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:16.154230 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:16.153483 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:16.251219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:16.250419 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded" Oct 30 15:56:16.315332 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:16.315282 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:16.548280 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:16.544280 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:16.648340 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:16.648291 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:16.668952 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:16.668912 5013 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": context deadline exceeded" Oct 30 15:56:17.221253 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:17.215963 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:17.221253 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:17.216638 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:56:17.271225 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:17.222288 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded" Oct 30 15:56:17.271225 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:17.246164 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:56:17.271225 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:17.251414 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded" Oct 30 15:56:17.271225 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:17.252511 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:56:17.331221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:17.327347 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:17.335218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:17.333755 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:56:17.556275 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:17.547053 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:17.565231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:17.558705 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:56:17.653621 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:17.653574 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:17.654772 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:17.654745 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:56:17.843402 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:17.819736 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:17.979217 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:17.979159 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:56:18.011745 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:18.010889 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:56:18.014554 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:18.012133 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:56:18.021652 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:18.016845 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:56:18.219358 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:18.216310 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:56:18.219358 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:18.216388 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:56:19.959691 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:19.959655 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:21.945474 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:21.945446 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:23.655368 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:23.655326 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:25.485788 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:25.485756 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:26.977052 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:56:26.977018 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:56:27.149479 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:27.149448 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:28.854072 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:28.854037 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:30.694494 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:30.694454 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:32.371383 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:32.371352 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:34.048630 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:34.048597 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:36.063114 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:36.062360 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:37.078218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:37.074782 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:37.138226 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:37.081558 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:56:37.517013 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:37.516984 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:56:37.740218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:37.739902 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:37.949342 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:37.949310 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:39.758217 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:39.758168 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:41.462978 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:41.462941 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:43.164554 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:43.164507 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:44.869634 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:44.869487 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:46.571530 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:46.571492 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:48.374437 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:48.374405 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:50.021381 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:50.021340 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:51.881850 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:51.881770 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:53.878731 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:53.878694 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:55.541508 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:55.541466 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:55.707936 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:55.707905 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:57.189433 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:56:57.189389 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:56:58.025002 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:58.024280 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:58.165221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:58.165117 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:56:58.719223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:58.708533 5013 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/health\": context deadline exceeded" Oct 30 15:56:59.093732 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:59.093617 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded" Oct 30 15:56:59.094582 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:59.094552 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:56:59.131229 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:59.109480 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:59.131229 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:59.124359 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded" Oct 30 15:56:59.131229 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:59.124892 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:59.222223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:59.196949 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:59.596220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:59.542292 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:56:59.872320 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:56:59.872120 5013 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:00.135280 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:00.126342 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded" Oct 30 15:57:00.135280 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:00.127696 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:00.154166 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:00.154123 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:00.178881 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:00.178462 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded" Oct 30 15:57:00.229531 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:00.229483 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:00.251234 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:00.240985 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:57:00.252379 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:00.252356 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:57:00.276292 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:00.276268 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:57:00.332354 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:00.332323 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:57:00.612627 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:00.612537 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded" Oct 30 15:57:00.613817 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:00.613796 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:57:01.068222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:01.067819 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:57:01.073706 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:01.073679 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:57:01.145138 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:01.143378 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:01.200650 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:01.200310 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded" Oct 30 15:57:01.232175 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:01.231886 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:01.687248 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:01.681416 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:02.155229 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:02.147291 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded" Oct 30 15:57:02.218247 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:02.204519 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:02.235339 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:02.235283 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:02.304484 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:02.304438 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:57:02.342432 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:02.342407 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:57:02.565746 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:02.565667 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:57:02.590477 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:02.590450 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:57:02.846689 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:02.846611 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:03.230904 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:03.230854 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:04.941489 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:04.941444 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:06.779773 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:06.779729 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:08.553404 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:08.553364 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:08.958238 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:08.958182 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29"] Oct 30 15:57:08.965078 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:08.965053 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:08.983558 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:08.983532 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29"] Oct 30 15:57:08.991096 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:08.991071 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-a2b3f1\"/\"distinct-w1-a2b3f1-kserve-self-signed-certs\"" Oct 30 15:57:09.045704 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.045673 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-tls-certs\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.045890 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.045723 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-dshm\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.046088 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.046060 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-home\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.046185 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.046167 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-tmp-dir\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.046273 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.046227 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x77gx\" (UniqueName: \"kubernetes.io/projected/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-kube-api-access-x77gx\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.046339 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.046319 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-model-cache\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.069830 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.069804 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:09.148021 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.147985 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-home\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.148235 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.148049 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-tmp-dir\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.148235 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.148079 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x77gx\" (UniqueName: \"kubernetes.io/projected/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-kube-api-access-x77gx\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.148235 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.148123 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-model-cache\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.148419 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.148265 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-tls-certs\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.148419 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.148323 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-dshm\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.148533 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.148448 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-home\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.148533 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.148487 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-tmp-dir\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.148699 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.148676 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-model-cache\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.152216 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.150989 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-dshm\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.152778 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.152743 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-tls-certs\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.185221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.180944 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x77gx\" (UniqueName: \"kubernetes.io/projected/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-kube-api-access-x77gx\") pod \"distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.284705 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.284604 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:09.609182 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:57:09.609158 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf0c6ebb_6ae6_455e_9b35_e72ffb5538ba.slice/crio-99b4c46cb9747cc5aee9cc06907bd1d5e05e3a98433e583b2750161754dbb7ac WatchSource:0}: Error finding container 99b4c46cb9747cc5aee9cc06907bd1d5e05e3a98433e583b2750161754dbb7ac: Status 404 returned error can't find the container with id 99b4c46cb9747cc5aee9cc06907bd1d5e05e3a98433e583b2750161754dbb7ac Oct 30 15:57:09.609943 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.609905 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29"] Oct 30 15:57:09.776140 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:09.776109 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:10.351232 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:10.347852 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:10.351232 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:10.350231 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" event={"ID":"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba","Type":"ContainerStarted","Data":"918c46e9d894d3330620e719f08439162581a0ba02668482f4810db3c39baefc"} Oct 30 15:57:10.351232 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:10.350262 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" event={"ID":"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba","Type":"ContainerStarted","Data":"99b4c46cb9747cc5aee9cc06907bd1d5e05e3a98433e583b2750161754dbb7ac"} Oct 30 15:57:10.392359 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:10.392308 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" podStartSLOduration=2.392294761 podStartE2EDuration="2.392294761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:57:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:57:10.391404987 +0000 UTC m=+1146.542302022" watchObservedRunningTime="2026-10-30 15:57:10.392294761 +0000 UTC m=+1146.543191792" Oct 30 15:57:11.285125 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:11.285087 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:12.304843 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:12.304800 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:14.022345 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:14.022311 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:15.850158 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:15.850113 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:17.724100 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:17.724070 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:19.285475 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:19.285442 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:19.298127 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:19.298097 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:19.423086 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:19.423052 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:19.661578 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:19.661543 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:21.441342 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:21.441310 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:23.249221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:23.247187 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:24.348534 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:24.347301 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:24.365213 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:24.361314 5013 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:24.416402 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:24.416342 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:24.603302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:24.598260 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded" Oct 30 15:57:25.080225 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:25.079324 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded" Oct 30 15:57:25.133221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:25.132286 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:25.149233 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:25.148771 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:25.213330 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:25.213283 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded" Oct 30 15:57:25.352654 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:25.352555 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:25.353890 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:25.353867 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:57:25.604456 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:25.604360 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:25.642218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:25.606486 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:57:25.772686 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:25.612934 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded" Oct 30 15:57:25.773120 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:25.773091 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:25.773603 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:25.682292 5013 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:26.112222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:26.109419 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded" Oct 30 15:57:26.204710 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:26.152835 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:26.204710 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:26.153130 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded" Oct 30 15:57:26.246338 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:26.246290 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:26.539417 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:26.493688 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:26.612219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:26.608707 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:26.635480 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:26.570619 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:57:26.635851 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:26.635829 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:57:26.635951 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:26.635917 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:57:26.636208 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:26.636174 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:57:26.791635 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:26.791530 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:26.820221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:26.796429 5013 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:27.153220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:27.129259 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:27.153711 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:27.153669 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:27.276218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:27.252362 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded" Oct 30 15:57:27.302272 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:27.285294 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:27.682274 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:27.609061 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded" Oct 30 15:57:27.743708 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:27.668395 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded" Oct 30 15:57:27.798225 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:27.797595 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:28.053508 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:57:28.053412 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:57:28.165536 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:28.164328 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:28.197223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:28.192577 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded" Oct 30 15:57:28.259216 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:28.254277 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded" Oct 30 15:57:28.290316 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:28.290271 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:28.380279 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:28.377638 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:28.631994 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:28.631909 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:57:28.748353 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:28.748309 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:28.749241 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:28.749210 5013 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:28.749442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:28.749417 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded" Oct 30 15:57:28.809690 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:28.807355 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded" Oct 30 15:57:28.970843 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:28.970758 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:57:29.039754 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:29.038483 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:57:29.039754 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:29.039657 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:57:29.079589 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:29.079559 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:57:29.097005 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:29.096976 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:57:29.099268 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:29.099175 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:57:29.126873 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:29.126847 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:31.084754 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:31.084721 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:31.441059 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:31.441027 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:33.269697 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:33.269667 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf"] Oct 30 15:57:33.434416 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:33.434385 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:33.604430 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:33.604352 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:35.259653 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:35.259617 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:36.298972 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:36.298940 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp"] Oct 30 15:57:36.463952 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:36.463923 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:37.036001 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:37.035968 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:38.794032 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:38.793995 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:41.126074 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:41.126032 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:41.916958 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:41.916925 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:42.104331 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.104293 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:42.703027 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.702996 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st"] Oct 30 15:57:42.707406 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.707387 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.723575 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.723555 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5"] Oct 30 15:57:42.727650 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.727629 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.742167 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.735891 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st"] Oct 30 15:57:42.754847 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.754233 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5"] Oct 30 15:57:42.799629 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.799594 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.799764 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.799704 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.799764 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.799732 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.799764 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.799758 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.799910 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.799796 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.799974 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.799922 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5hkh\" (UniqueName: \"kubernetes.io/projected/a06cb93e-6948-474e-bc0c-6601b10e1e3d-kube-api-access-v5hkh\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.800108 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.799971 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.800108 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.800025 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.800262 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.800146 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v75sv\" (UniqueName: \"kubernetes.io/projected/b477fa47-61b7-4232-a375-6ca81dfec7cf-kube-api-access-v75sv\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.800262 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.800245 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.800358 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.800287 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.800358 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.800315 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.800732 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.800709 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.800816 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.800745 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.800816 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.800775 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b477fa47-61b7-4232-a375-6ca81dfec7cf-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.800883 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.800828 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.800883 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.800866 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.800971 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.800924 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.904271 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.904225 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.904493 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.904308 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.904581 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.904550 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.904691 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.904650 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.904780 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.904692 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.904780 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.904728 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.904780 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.904770 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.904988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.904813 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5hkh\" (UniqueName: \"kubernetes.io/projected/a06cb93e-6948-474e-bc0c-6601b10e1e3d-kube-api-access-v5hkh\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.904988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.904875 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.904988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.904916 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.904988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.904966 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v75sv\" (UniqueName: \"kubernetes.io/projected/b477fa47-61b7-4232-a375-6ca81dfec7cf-kube-api-access-v75sv\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.905237 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.905034 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.905237 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.905078 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.905237 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.905109 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.905237 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.905160 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.905237 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.905219 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.905725 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.905435 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b477fa47-61b7-4232-a375-6ca81dfec7cf-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.905725 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.905472 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.905986 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.905963 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.906235 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.906216 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.906454 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.906277 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.907223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.906859 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.907223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.906875 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.907877 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.907744 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b477fa47-61b7-4232-a375-6ca81dfec7cf-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.908149 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.908129 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.908752 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.908704 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.909564 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.909542 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.910531 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.910509 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.910777 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.910759 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.911251 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.911215 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.911714 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.911689 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:42.912409 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.912381 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.921095 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.921034 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.928546 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.928517 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.928685 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.928594 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:42.928813 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.928799 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5hkh\" (UniqueName: \"kubernetes.io/projected/a06cb93e-6948-474e-bc0c-6601b10e1e3d-kube-api-access-v5hkh\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:42.931573 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:42.931547 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v75sv\" (UniqueName: \"kubernetes.io/projected/b477fa47-61b7-4232-a375-6ca81dfec7cf-kube-api-access-v75sv\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:43.022372 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.022285 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:43.045618 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.045597 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:43.349612 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:57:43.349586 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda06cb93e_6948_474e_bc0c_6601b10e1e3d.slice/crio-8b92ece03a82978189e9328386f5c58d340e24923c4cfd110c7da1b91f633126 WatchSource:0}: Error finding container 8b92ece03a82978189e9328386f5c58d340e24923c4cfd110c7da1b91f633126: Status 404 returned error can't find the container with id 8b92ece03a82978189e9328386f5c58d340e24923c4cfd110c7da1b91f633126 Oct 30 15:57:43.350404 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.350381 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st"] Oct 30 15:57:43.352020 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.351989 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:57:43.352102 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.352054 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:57:43.352102 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.352082 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:57:43.396496 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.396449 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:57:43.396621 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.396548 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:57:43.396621 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.396597 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:57:43.399633 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.399597 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5"] Oct 30 15:57:43.561338 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.561298 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" event={"ID":"b477fa47-61b7-4232-a375-6ca81dfec7cf","Type":"ContainerStarted","Data":"6b87d2e48f4eff69d48e3c76a6e861b31a743f3d3503bdd93566e17cc81f6216"} Oct 30 15:57:43.561675 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.561365 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" event={"ID":"b477fa47-61b7-4232-a375-6ca81dfec7cf","Type":"ContainerStarted","Data":"603ca00e72c07bd162330e7375f8328a397c5d7d6908703fa263159898d2d2bd"} Oct 30 15:57:43.567297 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.567267 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" event={"ID":"a06cb93e-6948-474e-bc0c-6601b10e1e3d","Type":"ContainerStarted","Data":"b8f302ae51f31a303cc92365bdbb551fb7dd8478a72ea6bf36d9b558237cf313"} Oct 30 15:57:43.567427 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.567325 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" event={"ID":"a06cb93e-6948-474e-bc0c-6601b10e1e3d","Type":"ContainerStarted","Data":"8b92ece03a82978189e9328386f5c58d340e24923c4cfd110c7da1b91f633126"} Oct 30 15:57:43.612728 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.612616 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" podStartSLOduration=1.612597378 podStartE2EDuration="1.612597378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:57:43.611733684 +0000 UTC m=+1179.762630726" watchObservedRunningTime="2026-10-30 15:57:43.612597378 +0000 UTC m=+1179.763494413" Oct 30 15:57:43.661416 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:43.661358 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" podStartSLOduration=1.661339673 podStartE2EDuration="1.661339673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:57:43.655992409 +0000 UTC m=+1179.806889439" watchObservedRunningTime="2026-10-30 15:57:43.661339673 +0000 UTC m=+1179.812236708" Oct 30 15:57:44.032403 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:44.032371 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 15:57:44.033012 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:44.032975 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" podUID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 15:57:44.037694 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:44.037670 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:44.046439 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:44.046418 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:44.047672 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:44.047646 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Oct 30 15:57:44.047785 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:44.047697 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" podUID="b477fa47-61b7-4232-a375-6ca81dfec7cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Oct 30 15:57:45.024640 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:45.024607 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 15:57:45.024902 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:45.024876 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" podUID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 15:57:45.046749 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:45.046715 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Oct 30 15:57:45.047339 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:45.047310 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" podUID="b477fa47-61b7-4232-a375-6ca81dfec7cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Oct 30 15:57:45.197514 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:45.193676 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:45.830833 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:45.830790 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t"] Oct 30 15:57:46.027163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:46.027088 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:46.027898 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:46.027873 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 15:57:46.028111 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:46.028085 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" podUID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 15:57:46.046897 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:46.046863 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Oct 30 15:57:46.047476 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:46.046924 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" podUID="b477fa47-61b7-4232-a375-6ca81dfec7cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Oct 30 15:57:46.203220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:46.196110 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:47.023628 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:47.023585 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 15:57:47.023923 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:47.023876 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" podUID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 15:57:47.046840 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:47.046802 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Oct 30 15:57:47.047067 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:47.047018 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" podUID="b477fa47-61b7-4232-a375-6ca81dfec7cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Oct 30 15:57:47.522522 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:47.522491 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:48.027381 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.027350 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 15:57:48.027630 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.027609 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" podUID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 15:57:48.046118 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.046082 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Oct 30 15:57:48.046292 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.046134 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" podUID="b477fa47-61b7-4232-a375-6ca81dfec7cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Oct 30 15:57:48.301907 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.300389 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" containerID="cri-o://65f0d73189509e41dcac2d0992620d304058ad9f5f77db7d289f873a2614e3b2" gracePeriod=45 Oct 30 15:57:48.669918 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.669874 5013 generic.go:356] "Generic (PLEG): container finished" podID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerID="65f0d73189509e41dcac2d0992620d304058ad9f5f77db7d289f873a2614e3b2" exitCode=0 Oct 30 15:57:48.670076 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.669923 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" event={"ID":"017d73bc-9fcc-4c13-bfdd-af21ac301a65","Type":"ContainerDied","Data":"65f0d73189509e41dcac2d0992620d304058ad9f5f77db7d289f873a2614e3b2"} Oct 30 15:57:48.845140 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.845091 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:57:48.948050 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.947964 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/017d73bc-9fcc-4c13-bfdd-af21ac301a65-tls-certs\" (UniqueName: \"kubernetes.io/secret/017d73bc-9fcc-4c13-bfdd-af21ac301a65-tls-certs\") pod \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " Oct 30 15:57:48.948707 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.948684 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-tmp-dir\") pod \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " Oct 30 15:57:48.949182 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.949161 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-model-cache\") pod \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " Oct 30 15:57:48.949378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.949360 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/017d73bc-9fcc-4c13-bfdd-af21ac301a65-kube-api-access-dmg7m\" (UniqueName: \"kubernetes.io/projected/017d73bc-9fcc-4c13-bfdd-af21ac301a65-kube-api-access-dmg7m\") pod \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " Oct 30 15:57:48.949633 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.949618 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-dshm\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-dshm\") pod \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " Oct 30 15:57:48.949785 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.949769 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-home\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-home\") pod \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\" (UID: \"017d73bc-9fcc-4c13-bfdd-af21ac301a65\") " Oct 30 15:57:48.950274 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.949003 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-tmp-dir" pod "017d73bc-9fcc-4c13-bfdd-af21ac301a65" (UID: "017d73bc-9fcc-4c13-bfdd-af21ac301a65"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:57:48.950369 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.949813 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-model-cache" pod "017d73bc-9fcc-4c13-bfdd-af21ac301a65" (UID: "017d73bc-9fcc-4c13-bfdd-af21ac301a65"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:57:48.950369 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.950233 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-home" pod "017d73bc-9fcc-4c13-bfdd-af21ac301a65" (UID: "017d73bc-9fcc-4c13-bfdd-af21ac301a65"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:57:48.950747 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.950729 5013 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-tmp-dir\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:57:48.950860 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.950847 5013 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-model-cache\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:57:48.950948 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.950936 5013 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-home\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:57:48.952254 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.952230 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/017d73bc-9fcc-4c13-bfdd-af21ac301a65-tls-certs" pod "017d73bc-9fcc-4c13-bfdd-af21ac301a65" (UID: "017d73bc-9fcc-4c13-bfdd-af21ac301a65"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:57:48.953757 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.953385 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-dshm" pod "017d73bc-9fcc-4c13-bfdd-af21ac301a65" (UID: "017d73bc-9fcc-4c13-bfdd-af21ac301a65"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:57:48.954264 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:48.954243 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/017d73bc-9fcc-4c13-bfdd-af21ac301a65-kube-api-access-dmg7m" pod "017d73bc-9fcc-4c13-bfdd-af21ac301a65" (UID: "017d73bc-9fcc-4c13-bfdd-af21ac301a65"). InnerVolumeSpecName "kube-api-access-dmg7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:57:49.037257 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:49.027411 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 15:57:49.037257 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:49.027473 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" podUID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 15:57:49.047988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:49.047955 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Oct 30 15:57:49.048164 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:49.048011 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" podUID="b477fa47-61b7-4232-a375-6ca81dfec7cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Oct 30 15:57:49.054900 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:49.051758 5013 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/017d73bc-9fcc-4c13-bfdd-af21ac301a65-tls-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:57:49.054900 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:49.051785 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmg7m\" (UniqueName: \"kubernetes.io/projected/017d73bc-9fcc-4c13-bfdd-af21ac301a65-kube-api-access-dmg7m\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:57:49.054900 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:49.051798 5013 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/017d73bc-9fcc-4c13-bfdd-af21ac301a65-dshm\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:57:49.679409 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:49.679376 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" Oct 30 15:57:49.679924 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:49.679381 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf" event={"ID":"017d73bc-9fcc-4c13-bfdd-af21ac301a65","Type":"ContainerDied","Data":"48a3fe27491c02f7f5c2865794555e1c88159eeabd055f4b263e0c405a2fa35a"} Oct 30 15:57:49.679924 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:49.679510 5013 scope.go:122] "RemoveContainer" containerID="65f0d73189509e41dcac2d0992620d304058ad9f5f77db7d289f873a2614e3b2" Oct 30 15:57:49.730540 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:49.730514 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf"] Oct 30 15:57:49.744231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:49.744177 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/distinct-w0-b63c3c-kserve-595dffc77c-nxrnf"] Oct 30 15:57:49.917979 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:49.917938 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:50.010309 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:50.010225 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" path="/var/lib/kubelet/pods/017d73bc-9fcc-4c13-bfdd-af21ac301a65/volumes" Oct 30 15:57:50.022939 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:50.022906 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 15:57:50.022939 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:50.022965 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" podUID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 15:57:50.046997 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:50.046962 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Oct 30 15:57:50.047141 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:50.047019 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" podUID="b477fa47-61b7-4232-a375-6ca81dfec7cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Oct 30 15:57:51.352220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:51.344530 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" containerID="cri-o://ecab5337e9406a8bb30208eb2473a9abf402401dbdbf0f53014329966f8281dc" gracePeriod=45 Oct 30 15:57:51.658260 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:51.655343 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:51.658260 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:51.656081 5013 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 15:57:51.658260 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:51.656142 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" podUID="0cb01099-f910-4f21-a0b6-ff5dbc33bf60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:51.658586 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:51.658281 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": dial tcp 10.132.0.56:8000: connect: connection refused" Oct 30 15:57:51.658586 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:51.658376 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:57:51.898446 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:51.881304 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 15:57:51.898446 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:51.881371 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" podUID="c0fda42f-3851-4542-a220-b9591e30d2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:51.979761 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:51.964344 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 15:57:51.979761 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:51.964410 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" podUID="1b325488-f2d5-49c8-9c76-d70554d9f0fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:51.995632 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:51.981474 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:52.028678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.028310 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 15:57:52.028678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.028364 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" podUID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:52.033342 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.033315 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:52.034908 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.034890 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:52.039569 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.039521 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:57:52.053658 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.053631 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 15:57:52.053874 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.053851 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" podUID="b477fa47-61b7-4232-a375-6ca81dfec7cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:52.064904 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.058600 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:52.103461 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.103239 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded" Oct 30 15:57:52.211863 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.200522 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:52.215660 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.215640 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:52.217892 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.217845 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:52.231905 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.229952 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:57:52.338914 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.336384 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 15:57:52.338914 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.336470 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" podUID="f0b22fc2-cafd-4d44-ab78-d1020db061fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 15:57:52.609858 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.609759 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": dial tcp 10.132.0.56:8000: connect: connection refused" Oct 30 15:57:52.700115 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.700077 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:52.883397 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.883365 5013 generic.go:356] "Generic (PLEG): container finished" podID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerID="ecab5337e9406a8bb30208eb2473a9abf402401dbdbf0f53014329966f8281dc" exitCode=0 Oct 30 15:57:52.886991 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:52.886931 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" event={"ID":"ae0e7089-01f3-42b4-9590-51cdc0b4765a","Type":"ContainerDied","Data":"ecab5337e9406a8bb30208eb2473a9abf402401dbdbf0f53014329966f8281dc"} Oct 30 15:57:53.025158 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.025127 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:53.294675 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.294644 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:57:53.310928 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.310901 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ae0e7089-01f3-42b4-9590-51cdc0b4765a-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae0e7089-01f3-42b4-9590-51cdc0b4765a-tls-certs\") pod \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " Oct 30 15:57:53.311057 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.310990 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae0e7089-01f3-42b4-9590-51cdc0b4765a-kube-api-access-bnx6p\" (UniqueName: \"kubernetes.io/projected/ae0e7089-01f3-42b4-9590-51cdc0b4765a-kube-api-access-bnx6p\") pod \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " Oct 30 15:57:53.311057 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.311018 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-dshm\") pod \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " Oct 30 15:57:53.311142 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.311060 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-tmp-dir\") pod \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " Oct 30 15:57:53.311228 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.311179 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-model-cache\") pod \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " Oct 30 15:57:53.311330 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.311309 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-home\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-home\") pod \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\" (UID: \"ae0e7089-01f3-42b4-9590-51cdc0b4765a\") " Oct 30 15:57:53.311519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.311440 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-model-cache" pod "ae0e7089-01f3-42b4-9590-51cdc0b4765a" (UID: "ae0e7089-01f3-42b4-9590-51cdc0b4765a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:57:53.311519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.311470 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-tmp-dir" pod "ae0e7089-01f3-42b4-9590-51cdc0b4765a" (UID: "ae0e7089-01f3-42b4-9590-51cdc0b4765a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:57:53.311607 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.311535 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-home" pod "ae0e7089-01f3-42b4-9590-51cdc0b4765a" (UID: "ae0e7089-01f3-42b4-9590-51cdc0b4765a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:57:53.311833 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.311817 5013 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-tmp-dir\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:57:53.311885 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.311838 5013 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-model-cache\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:57:53.311885 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.311849 5013 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-home\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:57:53.313632 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.313543 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae0e7089-01f3-42b4-9590-51cdc0b4765a-kube-api-access-bnx6p" pod "ae0e7089-01f3-42b4-9590-51cdc0b4765a" (UID: "ae0e7089-01f3-42b4-9590-51cdc0b4765a"). InnerVolumeSpecName "kube-api-access-bnx6p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:57:53.313767 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.313742 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae0e7089-01f3-42b4-9590-51cdc0b4765a-tls-certs" pod "ae0e7089-01f3-42b4-9590-51cdc0b4765a" (UID: "ae0e7089-01f3-42b4-9590-51cdc0b4765a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:57:53.313840 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.313821 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-dshm" pod "ae0e7089-01f3-42b4-9590-51cdc0b4765a" (UID: "ae0e7089-01f3-42b4-9590-51cdc0b4765a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:57:53.413983 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.413943 5013 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae0e7089-01f3-42b4-9590-51cdc0b4765a-tls-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:57:53.413983 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.413986 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnx6p\" (UniqueName: \"kubernetes.io/projected/ae0e7089-01f3-42b4-9590-51cdc0b4765a-kube-api-access-bnx6p\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:57:53.414234 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.413999 5013 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae0e7089-01f3-42b4-9590-51cdc0b4765a-dshm\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:57:53.510282 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.510024 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:53.900599 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.900555 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" event={"ID":"ae0e7089-01f3-42b4-9590-51cdc0b4765a","Type":"ContainerDied","Data":"4dfb219ce03d7a6a88cf48f7293262080c9dbc2d44d1372f2c51e064cfb4ce00"} Oct 30 15:57:53.901062 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.901043 5013 scope.go:122] "RemoveContainer" containerID="ecab5337e9406a8bb30208eb2473a9abf402401dbdbf0f53014329966f8281dc" Oct 30 15:57:53.903472 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.903447 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp" Oct 30 15:57:53.954815 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.954785 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp"] Oct 30 15:57:53.966293 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:53.966265 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-c129d1/simulator-w6-c129d1-kserve-b65d8b855-5h8dp"] Oct 30 15:57:54.012396 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:54.012354 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" path="/var/lib/kubelet/pods/ae0e7089-01f3-42b4-9590-51cdc0b4765a/volumes" Oct 30 15:57:55.020402 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:55.020361 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:55.652663 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:55.652629 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:55.813225 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:55.813177 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:55.975113 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:55.975038 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:56.375222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:56.375139 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:56.980087 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:57:56.980050 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:57:57.529111 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:57.529074 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:58.016628 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:58.016594 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:57:59.575926 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:57:59.575891 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:00.143472 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:00.143420 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:00.355033 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:00.354978 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:00.516004 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:00.515923 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:00.858006 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:00.857925 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" containerID="cri-o://2e5d4c345159d98344800fd31cc506a69c9b6267e4e921d75b1f9ae659d489d3" gracePeriod=45 Oct 30 15:58:00.918544 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:00.918514 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:01.126736 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.126679 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:58:01.231631 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.231602 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f9d63587-9839-45d5-8bbe-e5208b3446b6-tls-certs\" (UniqueName: \"kubernetes.io/secret/f9d63587-9839-45d5-8bbe-e5208b3446b6-tls-certs\") pod \"f9d63587-9839-45d5-8bbe-e5208b3446b6\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " Oct 30 15:58:01.231821 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.231722 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-home\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-home\") pod \"f9d63587-9839-45d5-8bbe-e5208b3446b6\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " Oct 30 15:58:01.231892 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.231827 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9d63587-9839-45d5-8bbe-e5208b3446b6-kube-api-access-ptgn8\" (UniqueName: \"kubernetes.io/projected/f9d63587-9839-45d5-8bbe-e5208b3446b6-kube-api-access-ptgn8\") pod \"f9d63587-9839-45d5-8bbe-e5208b3446b6\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " Oct 30 15:58:01.231892 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.231856 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-model-cache\") pod \"f9d63587-9839-45d5-8bbe-e5208b3446b6\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " Oct 30 15:58:01.231996 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.231902 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-dshm\") pod \"f9d63587-9839-45d5-8bbe-e5208b3446b6\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " Oct 30 15:58:01.231996 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.231954 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-tmp-dir\") pod \"f9d63587-9839-45d5-8bbe-e5208b3446b6\" (UID: \"f9d63587-9839-45d5-8bbe-e5208b3446b6\") " Oct 30 15:58:01.233500 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.232006 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-home" pod "f9d63587-9839-45d5-8bbe-e5208b3446b6" (UID: "f9d63587-9839-45d5-8bbe-e5208b3446b6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:01.233500 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.232425 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-tmp-dir" pod "f9d63587-9839-45d5-8bbe-e5208b3446b6" (UID: "f9d63587-9839-45d5-8bbe-e5208b3446b6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:01.233500 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.232752 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-model-cache" pod "f9d63587-9839-45d5-8bbe-e5208b3446b6" (UID: "f9d63587-9839-45d5-8bbe-e5208b3446b6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:01.233500 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.232856 5013 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-tmp-dir\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:01.233500 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.232899 5013 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-home\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:01.236544 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.234912 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-dshm" pod "f9d63587-9839-45d5-8bbe-e5208b3446b6" (UID: "f9d63587-9839-45d5-8bbe-e5208b3446b6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:01.237009 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.236952 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9d63587-9839-45d5-8bbe-e5208b3446b6-tls-certs" pod "f9d63587-9839-45d5-8bbe-e5208b3446b6" (UID: "f9d63587-9839-45d5-8bbe-e5208b3446b6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:58:01.237144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.237114 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9d63587-9839-45d5-8bbe-e5208b3446b6-kube-api-access-ptgn8" pod "f9d63587-9839-45d5-8bbe-e5208b3446b6" (UID: "f9d63587-9839-45d5-8bbe-e5208b3446b6"). InnerVolumeSpecName "kube-api-access-ptgn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:01.334110 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.334070 5013 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9d63587-9839-45d5-8bbe-e5208b3446b6-tls-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:01.334110 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.334110 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptgn8\" (UniqueName: \"kubernetes.io/projected/f9d63587-9839-45d5-8bbe-e5208b3446b6-kube-api-access-ptgn8\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:01.334323 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.334126 5013 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-model-cache\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:01.334323 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.334139 5013 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9d63587-9839-45d5-8bbe-e5208b3446b6-dshm\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:01.964255 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.964222 5013 generic.go:356] "Generic (PLEG): container finished" podID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerID="2e5d4c345159d98344800fd31cc506a69c9b6267e4e921d75b1f9ae659d489d3" exitCode=0 Oct 30 15:58:01.964683 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.964319 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" event={"ID":"f9d63587-9839-45d5-8bbe-e5208b3446b6","Type":"ContainerDied","Data":"2e5d4c345159d98344800fd31cc506a69c9b6267e4e921d75b1f9ae659d489d3"} Oct 30 15:58:01.964683 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.964335 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" Oct 30 15:58:01.964683 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.964357 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" event={"ID":"f9d63587-9839-45d5-8bbe-e5208b3446b6","Type":"ContainerDied","Data":"132eab97d0840e78c794cd0d94aa76bd59699f5acdc6f7d044dcf3c3e16e828f"} Oct 30 15:58:01.964683 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.964378 5013 scope.go:122] "RemoveContainer" containerID="2e5d4c345159d98344800fd31cc506a69c9b6267e4e921d75b1f9ae659d489d3" Oct 30 15:58:01.985065 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.985046 5013 scope.go:122] "RemoveContainer" containerID="2e5d4c345159d98344800fd31cc506a69c9b6267e4e921d75b1f9ae659d489d3" Oct 30 15:58:01.985385 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:58:01.985363 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e5d4c345159d98344800fd31cc506a69c9b6267e4e921d75b1f9ae659d489d3\": container with ID starting with 2e5d4c345159d98344800fd31cc506a69c9b6267e4e921d75b1f9ae659d489d3 not found: ID does not exist" containerID="2e5d4c345159d98344800fd31cc506a69c9b6267e4e921d75b1f9ae659d489d3" Oct 30 15:58:01.985447 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:01.985394 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e5d4c345159d98344800fd31cc506a69c9b6267e4e921d75b1f9ae659d489d3"} err="failed to get container status \"2e5d4c345159d98344800fd31cc506a69c9b6267e4e921d75b1f9ae659d489d3\": rpc error: code = NotFound desc = could not find container \"2e5d4c345159d98344800fd31cc506a69c9b6267e4e921d75b1f9ae659d489d3\": container with ID starting with 2e5d4c345159d98344800fd31cc506a69c9b6267e4e921d75b1f9ae659d489d3 not found: ID does not exist" Oct 30 15:58:02.016437 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:02.016414 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t"] Oct 30 15:58:02.027897 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:02.027871 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t"] Oct 30 15:58:02.080493 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:02.080461 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/distinct-2-w4-b24e7b-kserve-758df44f8d-lrq8t" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded" Oct 30 15:58:02.392408 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:02.392369 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:03.245519 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:03.245476 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:03.418987 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:03.418959 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:03.578297 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:03.568639 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww"] Oct 30 15:58:03.578297 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:03.568873 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" podUID="13d1b70a-d4d0-472d-b0fd-df159b4b3be3" containerName="istio-proxy" containerID="cri-o://9b02b5a5661a3133273423336faa9f56b5dbb96b16d09412399e048c9cdb02cd" gracePeriod=30 Oct 30 15:58:03.578854 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:03.578684 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6"] Oct 30 15:58:03.578915 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:03.578874 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" podUID="6f1ad3ab-f617-4a25-a2c3-c47ba625624f" containerName="istio-proxy" containerID="cri-o://e034790e8f537c1b3e443c50433aa934d1df5b7c06c5052d44a1de26423c0e7e" gracePeriod=30 Oct 30 15:58:03.605857 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:03.605829 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9"] Oct 30 15:58:03.606044 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:03.606023 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" podUID="04fbe630-342a-4ec9-91c6-48c89165b6de" containerName="istio-proxy" containerID="cri-o://3f41587f23dc811b0acffaf42eeb5e5124c10777aa08a7dd37bbeddd6fbee0ab" gracePeriod=30 Oct 30 15:58:03.677856 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:03.677813 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:04.008655 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:04.008616 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" path="/var/lib/kubelet/pods/f9d63587-9839-45d5-8bbe-e5208b3446b6/volumes" Oct 30 15:58:04.135038 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:04.135009 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 15:58:04.137822 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:04.137798 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 15:58:04.302719 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:04.302631 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:06.228800 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:06.228764 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:08.035038 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:08.034992 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:08.250222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:08.243656 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:08.295266 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:08.292773 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996"] Oct 30 15:58:08.295266 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:08.293573 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" podUID="0cb01099-f910-4f21-a0b6-ff5dbc33bf60" containerName="istio-proxy" containerID="cri-o://90543c34d4a9022aebfb8229aa56e3712f6323b98e0b34ad6e555068f3887c10" gracePeriod=30 Oct 30 15:58:08.337221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:08.336256 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c"] Oct 30 15:58:08.337221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:08.336658 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" podUID="46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" containerName="istio-proxy" containerID="cri-o://e497f068bb92fdfcb5f000d2e2e08ed0a37856789aa3ec3c702092def1545b21" gracePeriod=30 Oct 30 15:58:08.347771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:08.347735 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8"] Oct 30 15:58:08.347976 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:08.347950 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" podUID="83bf7cb0-3eae-41b2-8ba0-4db7b4634071" containerName="istio-proxy" containerID="cri-o://788de54dd368efd887bbd708c8e08a82b2114bdf44f2cdc0e6d3912505afa07b" gracePeriod=30 Oct 30 15:58:08.507220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:08.497730 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:09.026974 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.026942 5013 generic.go:356] "Generic (PLEG): container finished" podID="04fbe630-342a-4ec9-91c6-48c89165b6de" containerID="3f41587f23dc811b0acffaf42eeb5e5124c10777aa08a7dd37bbeddd6fbee0ab" exitCode=0 Oct 30 15:58:09.027252 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.027225 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" event={"ID":"04fbe630-342a-4ec9-91c6-48c89165b6de","Type":"ContainerDied","Data":"3f41587f23dc811b0acffaf42eeb5e5124c10777aa08a7dd37bbeddd6fbee0ab"} Oct 30 15:58:09.029650 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.029626 5013 generic.go:356] "Generic (PLEG): container finished" podID="13d1b70a-d4d0-472d-b0fd-df159b4b3be3" containerID="9b02b5a5661a3133273423336faa9f56b5dbb96b16d09412399e048c9cdb02cd" exitCode=0 Oct 30 15:58:09.029924 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.029902 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" event={"ID":"13d1b70a-d4d0-472d-b0fd-df159b4b3be3","Type":"ContainerDied","Data":"9b02b5a5661a3133273423336faa9f56b5dbb96b16d09412399e048c9cdb02cd"} Oct 30 15:58:09.032258 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.032236 5013 generic.go:356] "Generic (PLEG): container finished" podID="6f1ad3ab-f617-4a25-a2c3-c47ba625624f" containerID="e034790e8f537c1b3e443c50433aa934d1df5b7c06c5052d44a1de26423c0e7e" exitCode=0 Oct 30 15:58:09.032578 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.032554 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" event={"ID":"6f1ad3ab-f617-4a25-a2c3-c47ba625624f","Type":"ContainerDied","Data":"e034790e8f537c1b3e443c50433aa934d1df5b7c06c5052d44a1de26423c0e7e"} Oct 30 15:58:09.032725 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.032710 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" event={"ID":"6f1ad3ab-f617-4a25-a2c3-c47ba625624f","Type":"ContainerDied","Data":"a3ea535cc9cf68316e1a552c6f7bac3a6eaaa0232204c459a36fa544f88d4a7f"} Oct 30 15:58:09.032983 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.032967 5013 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a3ea535cc9cf68316e1a552c6f7bac3a6eaaa0232204c459a36fa544f88d4a7f" Oct 30 15:58:09.068688 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.068668 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:58:09.119745 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.119721 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:58:09.135240 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.135216 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:58:09.160882 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.160852 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istiod-ca-cert\") pod \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " Oct 30 15:58:09.161057 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.160967 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-credential-socket\") pod \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " Oct 30 15:58:09.161057 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.161012 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-workload-socket\") pod \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " Oct 30 15:58:09.161057 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.161045 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-token\" (UniqueName: \"kubernetes.io/projected/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-token\") pod \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " Oct 30 15:58:09.161286 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.161102 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-workload-certs\") pod \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " Oct 30 15:58:09.161286 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.161135 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-podinfo\") pod \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " Oct 30 15:58:09.161286 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.161245 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-envoy\") pod \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " Oct 30 15:58:09.161286 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.161281 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-kube-api-access-qrmwx\" (UniqueName: \"kubernetes.io/projected/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-kube-api-access-qrmwx\") pod \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " Oct 30 15:58:09.161499 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.161425 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-data\") pod \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\" (UID: \"6f1ad3ab-f617-4a25-a2c3-c47ba625624f\") " Oct 30 15:58:09.162656 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.162130 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-data" pod "6f1ad3ab-f617-4a25-a2c3-c47ba625624f" (UID: "6f1ad3ab-f617-4a25-a2c3-c47ba625624f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.162656 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.162262 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-credential-socket" pod "6f1ad3ab-f617-4a25-a2c3-c47ba625624f" (UID: "6f1ad3ab-f617-4a25-a2c3-c47ba625624f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.162656 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.162356 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-workload-certs" pod "6f1ad3ab-f617-4a25-a2c3-c47ba625624f" (UID: "6f1ad3ab-f617-4a25-a2c3-c47ba625624f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.162656 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.162368 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-workload-socket" pod "6f1ad3ab-f617-4a25-a2c3-c47ba625624f" (UID: "6f1ad3ab-f617-4a25-a2c3-c47ba625624f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.162656 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.162628 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istiod-ca-cert" pod "6f1ad3ab-f617-4a25-a2c3-c47ba625624f" (UID: "6f1ad3ab-f617-4a25-a2c3-c47ba625624f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:58:09.164948 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.164917 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-envoy" pod "6f1ad3ab-f617-4a25-a2c3-c47ba625624f" (UID: "6f1ad3ab-f617-4a25-a2c3-c47ba625624f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.166730 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.166700 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-podinfo" pod "6f1ad3ab-f617-4a25-a2c3-c47ba625624f" (UID: "6f1ad3ab-f617-4a25-a2c3-c47ba625624f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:58:09.167460 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.167425 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-kube-api-access-qrmwx" pod "6f1ad3ab-f617-4a25-a2c3-c47ba625624f" (UID: "6f1ad3ab-f617-4a25-a2c3-c47ba625624f"). InnerVolumeSpecName "kube-api-access-qrmwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:09.169962 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.169940 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-token" pod "6f1ad3ab-f617-4a25-a2c3-c47ba625624f" (UID: "6f1ad3ab-f617-4a25-a2c3-c47ba625624f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:09.263350 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.263265 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-kube-api-access-b6qwc\" (UniqueName: \"kubernetes.io/projected/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-kube-api-access-b6qwc\") pod \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " Oct 30 15:58:09.263350 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.263312 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-workload-certs\") pod \"04fbe630-342a-4ec9-91c6-48c89165b6de\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " Oct 30 15:58:09.263350 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.263341 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-token\" (UniqueName: \"kubernetes.io/projected/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-token\") pod \"04fbe630-342a-4ec9-91c6-48c89165b6de\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " Oct 30 15:58:09.263648 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.263589 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-envoy\") pod \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " Oct 30 15:58:09.263718 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.263660 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04fbe630-342a-4ec9-91c6-48c89165b6de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04fbe630-342a-4ec9-91c6-48c89165b6de-istiod-ca-cert\") pod \"04fbe630-342a-4ec9-91c6-48c89165b6de\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " Oct 30 15:58:09.263718 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.263699 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04fbe630-342a-4ec9-91c6-48c89165b6de-kube-api-access-scxr7\" (UniqueName: \"kubernetes.io/projected/04fbe630-342a-4ec9-91c6-48c89165b6de-kube-api-access-scxr7\") pod \"04fbe630-342a-4ec9-91c6-48c89165b6de\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " Oct 30 15:58:09.263831 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.263718 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-workload-certs" pod "04fbe630-342a-4ec9-91c6-48c89165b6de" (UID: "04fbe630-342a-4ec9-91c6-48c89165b6de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.263831 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.263744 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-podinfo\") pod \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " Oct 30 15:58:09.263831 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.263823 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-workload-certs\") pod \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " Oct 30 15:58:09.263988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.263874 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-workload-socket\") pod \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " Oct 30 15:58:09.263988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.263904 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-data\") pod \"04fbe630-342a-4ec9-91c6-48c89165b6de\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " Oct 30 15:58:09.263988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.263965 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-data\") pod \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " Oct 30 15:58:09.264144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.263999 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-envoy\") pod \"04fbe630-342a-4ec9-91c6-48c89165b6de\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " Oct 30 15:58:09.264144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.264010 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04fbe630-342a-4ec9-91c6-48c89165b6de-istiod-ca-cert" pod "04fbe630-342a-4ec9-91c6-48c89165b6de" (UID: "04fbe630-342a-4ec9-91c6-48c89165b6de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:58:09.264144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.264051 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-credential-socket\") pod \"04fbe630-342a-4ec9-91c6-48c89165b6de\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " Oct 30 15:58:09.264144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.264088 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-credential-socket\") pod \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " Oct 30 15:58:09.264144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.264128 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-token\" (UniqueName: \"kubernetes.io/projected/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-token\") pod \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " Oct 30 15:58:09.264415 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.264155 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-podinfo\") pod \"04fbe630-342a-4ec9-91c6-48c89165b6de\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " Oct 30 15:58:09.264415 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.264183 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-workload-certs" pod "13d1b70a-d4d0-472d-b0fd-df159b4b3be3" (UID: "13d1b70a-d4d0-472d-b0fd-df159b4b3be3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.264415 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.264222 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-workload-socket\") pod \"04fbe630-342a-4ec9-91c6-48c89165b6de\" (UID: \"04fbe630-342a-4ec9-91c6-48c89165b6de\") " Oct 30 15:58:09.264415 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.264328 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istiod-ca-cert\") pod \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\" (UID: \"13d1b70a-d4d0-472d-b0fd-df159b4b3be3\") " Oct 30 15:58:09.265181 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.265157 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.265380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.265185 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.265380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.265223 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.265380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.265236 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.265380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.265250 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrmwx\" (UniqueName: \"kubernetes.io/projected/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-kube-api-access-qrmwx\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.265380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.265265 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.265380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.265278 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.265380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.265290 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.265380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.265303 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04fbe630-342a-4ec9-91c6-48c89165b6de-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.265380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.265315 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.265380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.265330 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.265380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.265343 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f1ad3ab-f617-4a25-a2c3-c47ba625624f-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.266121 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.265968 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istiod-ca-cert" pod "13d1b70a-d4d0-472d-b0fd-df159b4b3be3" (UID: "13d1b70a-d4d0-472d-b0fd-df159b4b3be3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:58:09.266121 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.266062 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-credential-socket" pod "04fbe630-342a-4ec9-91c6-48c89165b6de" (UID: "04fbe630-342a-4ec9-91c6-48c89165b6de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.266358 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.266327 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-workload-socket" pod "13d1b70a-d4d0-472d-b0fd-df159b4b3be3" (UID: "13d1b70a-d4d0-472d-b0fd-df159b4b3be3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.266522 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.266506 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-data" pod "04fbe630-342a-4ec9-91c6-48c89165b6de" (UID: "04fbe630-342a-4ec9-91c6-48c89165b6de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.266712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.266649 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-data" pod "13d1b70a-d4d0-472d-b0fd-df159b4b3be3" (UID: "13d1b70a-d4d0-472d-b0fd-df159b4b3be3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.266969 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.266951 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04fbe630-342a-4ec9-91c6-48c89165b6de-kube-api-access-scxr7" pod "04fbe630-342a-4ec9-91c6-48c89165b6de" (UID: "04fbe630-342a-4ec9-91c6-48c89165b6de"). InnerVolumeSpecName "kube-api-access-scxr7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:09.267529 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.267503 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-credential-socket" pod "13d1b70a-d4d0-472d-b0fd-df159b4b3be3" (UID: "13d1b70a-d4d0-472d-b0fd-df159b4b3be3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.267905 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.267712 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-workload-socket" pod "04fbe630-342a-4ec9-91c6-48c89165b6de" (UID: "04fbe630-342a-4ec9-91c6-48c89165b6de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.267905 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.267745 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-kube-api-access-b6qwc" pod "13d1b70a-d4d0-472d-b0fd-df159b4b3be3" (UID: "13d1b70a-d4d0-472d-b0fd-df159b4b3be3"). InnerVolumeSpecName "kube-api-access-b6qwc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:09.267905 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.267825 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-envoy" pod "13d1b70a-d4d0-472d-b0fd-df159b4b3be3" (UID: "13d1b70a-d4d0-472d-b0fd-df159b4b3be3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.268918 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.268879 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-podinfo" pod "13d1b70a-d4d0-472d-b0fd-df159b4b3be3" (UID: "13d1b70a-d4d0-472d-b0fd-df159b4b3be3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:58:09.269223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.269179 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-envoy" pod "04fbe630-342a-4ec9-91c6-48c89165b6de" (UID: "04fbe630-342a-4ec9-91c6-48c89165b6de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:09.269562 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.269535 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-token" pod "04fbe630-342a-4ec9-91c6-48c89165b6de" (UID: "04fbe630-342a-4ec9-91c6-48c89165b6de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:09.270102 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.270075 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-token" pod "13d1b70a-d4d0-472d-b0fd-df159b4b3be3" (UID: "13d1b70a-d4d0-472d-b0fd-df159b4b3be3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:09.270566 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.270541 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-podinfo" pod "04fbe630-342a-4ec9-91c6-48c89165b6de" (UID: "04fbe630-342a-4ec9-91c6-48c89165b6de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:58:09.366698 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366663 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.366698 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366703 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.366903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366718 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.366903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366732 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.366903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366746 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.366903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366758 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.366903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366772 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6qwc\" (UniqueName: \"kubernetes.io/projected/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-kube-api-access-b6qwc\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.366903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366786 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.366903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366799 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.366903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366812 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scxr7\" (UniqueName: \"kubernetes.io/projected/04fbe630-342a-4ec9-91c6-48c89165b6de-kube-api-access-scxr7\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.366903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366825 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.366903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366837 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.366903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366848 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.366903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366877 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13d1b70a-d4d0-472d-b0fd-df159b4b3be3-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:09.366903 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:09.366890 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04fbe630-342a-4ec9-91c6-48c89165b6de-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:10.043379 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.043352 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" Oct 30 15:58:10.043558 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.043342 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9" event={"ID":"04fbe630-342a-4ec9-91c6-48c89165b6de","Type":"ContainerDied","Data":"a913cec80bc481b4875bbdfbb94090495fc9676c6457d1ffcd19551958749d1c"} Oct 30 15:58:10.043693 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.043634 5013 scope.go:122] "RemoveContainer" containerID="3f41587f23dc811b0acffaf42eeb5e5124c10777aa08a7dd37bbeddd6fbee0ab" Oct 30 15:58:10.045506 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.045464 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" event={"ID":"13d1b70a-d4d0-472d-b0fd-df159b4b3be3","Type":"ContainerDied","Data":"8164bbe096f9221df24a9ef92f43afedcdca88dfdf498440fd0543d1ce82ed31"} Oct 30 15:58:10.045604 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.045524 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww" Oct 30 15:58:10.045797 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.045778 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6" Oct 30 15:58:10.057910 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.057889 5013 scope.go:122] "RemoveContainer" containerID="9b02b5a5661a3133273423336faa9f56b5dbb96b16d09412399e048c9cdb02cd" Oct 30 15:58:10.096137 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.096111 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww"] Oct 30 15:58:10.109976 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.109948 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-twzww"] Oct 30 15:58:10.146769 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.146743 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9"] Oct 30 15:58:10.151902 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.151877 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-56hn9"] Oct 30 15:58:10.184884 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.184861 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6"] Oct 30 15:58:10.190284 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.190264 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-7vjd6"] Oct 30 15:58:10.417251 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.417181 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:10.751063 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.750985 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:10.923887 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:10.923860 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:11.087459 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:11.087371 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:12.007285 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:12.007258 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04fbe630-342a-4ec9-91c6-48c89165b6de" path="/var/lib/kubelet/pods/04fbe630-342a-4ec9-91c6-48c89165b6de/volumes" Oct 30 15:58:12.007772 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:12.007757 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13d1b70a-d4d0-472d-b0fd-df159b4b3be3" path="/var/lib/kubelet/pods/13d1b70a-d4d0-472d-b0fd-df159b4b3be3/volumes" Oct 30 15:58:12.008248 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:12.008229 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f1ad3ab-f617-4a25-a2c3-c47ba625624f" path="/var/lib/kubelet/pods/6f1ad3ab-f617-4a25-a2c3-c47ba625624f/volumes" Oct 30 15:58:12.145464 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:12.145435 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:12.385186 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:12.385147 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:13.656959 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.656929 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:58:13.714426 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.714398 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:58:13.721511 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.721484 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:58:13.822014 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.821937 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-token\" (UniqueName: \"kubernetes.io/projected/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-token\") pod \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " Oct 30 15:58:13.822014 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822003 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-envoy\") pod \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " Oct 30 15:58:13.822242 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822048 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-podinfo\") pod \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " Oct 30 15:58:13.822242 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822089 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istiod-ca-cert\") pod \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " Oct 30 15:58:13.822242 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822121 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-credential-socket\") pod \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " Oct 30 15:58:13.822242 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822152 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-podinfo\") pod \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " Oct 30 15:58:13.822242 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822224 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-podinfo\") pod \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " Oct 30 15:58:13.822498 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822250 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-workload-certs\") pod \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " Oct 30 15:58:13.822498 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822281 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-data\") pod \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " Oct 30 15:58:13.822498 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822297 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-kube-api-access-5vn6r\" (UniqueName: \"kubernetes.io/projected/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-kube-api-access-5vn6r\") pod \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " Oct 30 15:58:13.822498 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822318 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istiod-ca-cert\") pod \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " Oct 30 15:58:13.822498 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822345 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-kube-api-access-9skdx\" (UniqueName: \"kubernetes.io/projected/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-kube-api-access-9skdx\") pod \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " Oct 30 15:58:13.822498 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822384 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-data\") pod \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " Oct 30 15:58:13.822498 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822403 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-envoy\") pod \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " Oct 30 15:58:13.822498 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822468 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-token\" (UniqueName: \"kubernetes.io/projected/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-token\") pod \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " Oct 30 15:58:13.822498 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822484 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-workload-socket\") pod \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " Oct 30 15:58:13.822931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822506 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-workload-certs\") pod \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " Oct 30 15:58:13.822931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822540 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-credential-socket\") pod \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " Oct 30 15:58:13.822931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822558 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-envoy\") pod \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " Oct 30 15:58:13.822931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822591 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-workload-socket\") pod \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " Oct 30 15:58:13.822931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822636 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-data\") pod \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " Oct 30 15:58:13.822931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822684 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istiod-ca-cert\") pod \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " Oct 30 15:58:13.822931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822727 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-workload-certs\") pod \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\" (UID: \"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92\") " Oct 30 15:58:13.822931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822761 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-workload-socket\") pod \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " Oct 30 15:58:13.822931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822789 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-token\" (UniqueName: \"kubernetes.io/projected/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-token\") pod \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " Oct 30 15:58:13.822931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822875 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-credential-socket\") pod \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\" (UID: \"0cb01099-f910-4f21-a0b6-ff5dbc33bf60\") " Oct 30 15:58:13.823998 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.822935 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-kube-api-access-gnbn6\" (UniqueName: \"kubernetes.io/projected/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-kube-api-access-gnbn6\") pod \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\" (UID: \"83bf7cb0-3eae-41b2-8ba0-4db7b4634071\") " Oct 30 15:58:13.826028 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.825995 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-data" pod "0cb01099-f910-4f21-a0b6-ff5dbc33bf60" (UID: "0cb01099-f910-4f21-a0b6-ff5dbc33bf60"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.827029 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.827002 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-workload-socket" pod "83bf7cb0-3eae-41b2-8ba0-4db7b4634071" (UID: "83bf7cb0-3eae-41b2-8ba0-4db7b4634071"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.828302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.827318 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-workload-certs" pod "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" (UID: "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.828302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.827568 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-data" pod "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" (UID: "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.828302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.827714 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-credential-socket" pod "0cb01099-f910-4f21-a0b6-ff5dbc33bf60" (UID: "0cb01099-f910-4f21-a0b6-ff5dbc33bf60"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.828302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.827937 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-workload-certs" pod "83bf7cb0-3eae-41b2-8ba0-4db7b4634071" (UID: "83bf7cb0-3eae-41b2-8ba0-4db7b4634071"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.828302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.828068 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-workload-socket" pod "0cb01099-f910-4f21-a0b6-ff5dbc33bf60" (UID: "0cb01099-f910-4f21-a0b6-ff5dbc33bf60"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.828302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.828152 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-workload-certs" pod "0cb01099-f910-4f21-a0b6-ff5dbc33bf60" (UID: "0cb01099-f910-4f21-a0b6-ff5dbc33bf60"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.828676 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.828406 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-data" pod "83bf7cb0-3eae-41b2-8ba0-4db7b4634071" (UID: "83bf7cb0-3eae-41b2-8ba0-4db7b4634071"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.828676 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.828463 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-workload-socket" pod "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" (UID: "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.828676 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.828570 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istiod-ca-cert" pod "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" (UID: "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:58:13.829724 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.829025 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-credential-socket" pod "83bf7cb0-3eae-41b2-8ba0-4db7b4634071" (UID: "83bf7cb0-3eae-41b2-8ba0-4db7b4634071"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.829724 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.829388 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istiod-ca-cert" pod "0cb01099-f910-4f21-a0b6-ff5dbc33bf60" (UID: "0cb01099-f910-4f21-a0b6-ff5dbc33bf60"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:58:13.829724 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.829530 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-credential-socket" pod "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" (UID: "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.830184 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.830094 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istiod-ca-cert" pod "83bf7cb0-3eae-41b2-8ba0-4db7b4634071" (UID: "83bf7cb0-3eae-41b2-8ba0-4db7b4634071"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:58:13.832290 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.832041 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-envoy" pod "83bf7cb0-3eae-41b2-8ba0-4db7b4634071" (UID: "83bf7cb0-3eae-41b2-8ba0-4db7b4634071"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.832290 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.832176 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-token" pod "0cb01099-f910-4f21-a0b6-ff5dbc33bf60" (UID: "0cb01099-f910-4f21-a0b6-ff5dbc33bf60"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:13.832462 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.832301 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-envoy" pod "0cb01099-f910-4f21-a0b6-ff5dbc33bf60" (UID: "0cb01099-f910-4f21-a0b6-ff5dbc33bf60"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.839470 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.839434 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-podinfo" pod "0cb01099-f910-4f21-a0b6-ff5dbc33bf60" (UID: "0cb01099-f910-4f21-a0b6-ff5dbc33bf60"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:58:13.839558 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.839476 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-token" pod "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" (UID: "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:13.839558 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.839496 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-token" pod "83bf7cb0-3eae-41b2-8ba0-4db7b4634071" (UID: "83bf7cb0-3eae-41b2-8ba0-4db7b4634071"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:13.839664 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.839584 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-kube-api-access-9skdx" pod "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" (UID: "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92"). InnerVolumeSpecName "kube-api-access-9skdx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:13.839664 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.839609 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-kube-api-access-5vn6r" pod "0cb01099-f910-4f21-a0b6-ff5dbc33bf60" (UID: "0cb01099-f910-4f21-a0b6-ff5dbc33bf60"). InnerVolumeSpecName "kube-api-access-5vn6r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:13.839756 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.839692 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-envoy" pod "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" (UID: "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:13.839756 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.839723 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-kube-api-access-gnbn6" pod "83bf7cb0-3eae-41b2-8ba0-4db7b4634071" (UID: "83bf7cb0-3eae-41b2-8ba0-4db7b4634071"). InnerVolumeSpecName "kube-api-access-gnbn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:13.839756 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.839731 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-podinfo" pod "83bf7cb0-3eae-41b2-8ba0-4db7b4634071" (UID: "83bf7cb0-3eae-41b2-8ba0-4db7b4634071"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:58:13.841354 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.841305 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-podinfo" pod "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" (UID: "46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:58:13.925222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925177 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925221 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925232 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925243 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925251 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vn6r\" (UniqueName: \"kubernetes.io/projected/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-kube-api-access-5vn6r\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925261 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925269 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9skdx\" (UniqueName: \"kubernetes.io/projected/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-kube-api-access-9skdx\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925280 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925288 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925295 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925302 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925309 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925317 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925325 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925334 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925341 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925350 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925358 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925366 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925374 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925382 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925390 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnbn6\" (UniqueName: \"kubernetes.io/projected/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-kube-api-access-gnbn6\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925398 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83bf7cb0-3eae-41b2-8ba0-4db7b4634071-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925407 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925414 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925421 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cb01099-f910-4f21-a0b6-ff5dbc33bf60-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:13.925442 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:13.925428 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:14.091913 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.091832 5013 generic.go:356] "Generic (PLEG): container finished" podID="0cb01099-f910-4f21-a0b6-ff5dbc33bf60" containerID="90543c34d4a9022aebfb8229aa56e3712f6323b98e0b34ad6e555068f3887c10" exitCode=0 Oct 30 15:58:14.092080 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.091936 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" Oct 30 15:58:14.092080 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.091932 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" event={"ID":"0cb01099-f910-4f21-a0b6-ff5dbc33bf60","Type":"ContainerDied","Data":"90543c34d4a9022aebfb8229aa56e3712f6323b98e0b34ad6e555068f3887c10"} Oct 30 15:58:14.092080 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.092001 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996" event={"ID":"0cb01099-f910-4f21-a0b6-ff5dbc33bf60","Type":"ContainerDied","Data":"bb16090180f849648455bca0b9708221cd670a7647dc3810c8fca1bf0327173e"} Oct 30 15:58:14.092080 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.092033 5013 scope.go:122] "RemoveContainer" containerID="90543c34d4a9022aebfb8229aa56e3712f6323b98e0b34ad6e555068f3887c10" Oct 30 15:58:14.093864 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.093844 5013 generic.go:356] "Generic (PLEG): container finished" podID="83bf7cb0-3eae-41b2-8ba0-4db7b4634071" containerID="788de54dd368efd887bbd708c8e08a82b2114bdf44f2cdc0e6d3912505afa07b" exitCode=0 Oct 30 15:58:14.094049 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.093876 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" event={"ID":"83bf7cb0-3eae-41b2-8ba0-4db7b4634071","Type":"ContainerDied","Data":"788de54dd368efd887bbd708c8e08a82b2114bdf44f2cdc0e6d3912505afa07b"} Oct 30 15:58:14.094243 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.094079 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" event={"ID":"83bf7cb0-3eae-41b2-8ba0-4db7b4634071","Type":"ContainerDied","Data":"8a861b7c6531ffc3ad89ef275ec9cc0ed402d2d42b934b3eeee96334dd74f4b6"} Oct 30 15:58:14.094243 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.093893 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8" Oct 30 15:58:14.096256 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.096231 5013 generic.go:356] "Generic (PLEG): container finished" podID="46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" containerID="e497f068bb92fdfcb5f000d2e2e08ed0a37856789aa3ec3c702092def1545b21" exitCode=0 Oct 30 15:58:14.096367 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.096284 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" event={"ID":"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92","Type":"ContainerDied","Data":"e497f068bb92fdfcb5f000d2e2e08ed0a37856789aa3ec3c702092def1545b21"} Oct 30 15:58:14.096367 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.096325 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" event={"ID":"46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92","Type":"ContainerDied","Data":"577cc67fe9df1a1221022867a2666448d7f0690cc3944fa63d7c0b3c00a69875"} Oct 30 15:58:14.096367 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.096271 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c" Oct 30 15:58:14.105052 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.105038 5013 scope.go:122] "RemoveContainer" containerID="90543c34d4a9022aebfb8229aa56e3712f6323b98e0b34ad6e555068f3887c10" Oct 30 15:58:14.105356 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:58:14.105334 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90543c34d4a9022aebfb8229aa56e3712f6323b98e0b34ad6e555068f3887c10\": container with ID starting with 90543c34d4a9022aebfb8229aa56e3712f6323b98e0b34ad6e555068f3887c10 not found: ID does not exist" containerID="90543c34d4a9022aebfb8229aa56e3712f6323b98e0b34ad6e555068f3887c10" Oct 30 15:58:14.105429 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.105366 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90543c34d4a9022aebfb8229aa56e3712f6323b98e0b34ad6e555068f3887c10"} err="failed to get container status \"90543c34d4a9022aebfb8229aa56e3712f6323b98e0b34ad6e555068f3887c10\": rpc error: code = NotFound desc = could not find container \"90543c34d4a9022aebfb8229aa56e3712f6323b98e0b34ad6e555068f3887c10\": container with ID starting with 90543c34d4a9022aebfb8229aa56e3712f6323b98e0b34ad6e555068f3887c10 not found: ID does not exist" Oct 30 15:58:14.105429 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.105382 5013 scope.go:122] "RemoveContainer" containerID="788de54dd368efd887bbd708c8e08a82b2114bdf44f2cdc0e6d3912505afa07b" Oct 30 15:58:14.117000 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.116983 5013 scope.go:122] "RemoveContainer" containerID="788de54dd368efd887bbd708c8e08a82b2114bdf44f2cdc0e6d3912505afa07b" Oct 30 15:58:14.117344 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:58:14.117315 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"788de54dd368efd887bbd708c8e08a82b2114bdf44f2cdc0e6d3912505afa07b\": container with ID starting with 788de54dd368efd887bbd708c8e08a82b2114bdf44f2cdc0e6d3912505afa07b not found: ID does not exist" containerID="788de54dd368efd887bbd708c8e08a82b2114bdf44f2cdc0e6d3912505afa07b" Oct 30 15:58:14.117411 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.117353 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"788de54dd368efd887bbd708c8e08a82b2114bdf44f2cdc0e6d3912505afa07b"} err="failed to get container status \"788de54dd368efd887bbd708c8e08a82b2114bdf44f2cdc0e6d3912505afa07b\": rpc error: code = NotFound desc = could not find container \"788de54dd368efd887bbd708c8e08a82b2114bdf44f2cdc0e6d3912505afa07b\": container with ID starting with 788de54dd368efd887bbd708c8e08a82b2114bdf44f2cdc0e6d3912505afa07b not found: ID does not exist" Oct 30 15:58:14.117411 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.117370 5013 scope.go:122] "RemoveContainer" containerID="e497f068bb92fdfcb5f000d2e2e08ed0a37856789aa3ec3c702092def1545b21" Oct 30 15:58:14.130598 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.130556 5013 scope.go:122] "RemoveContainer" containerID="e497f068bb92fdfcb5f000d2e2e08ed0a37856789aa3ec3c702092def1545b21" Oct 30 15:58:14.130870 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:58:14.130850 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e497f068bb92fdfcb5f000d2e2e08ed0a37856789aa3ec3c702092def1545b21\": container with ID starting with e497f068bb92fdfcb5f000d2e2e08ed0a37856789aa3ec3c702092def1545b21 not found: ID does not exist" containerID="e497f068bb92fdfcb5f000d2e2e08ed0a37856789aa3ec3c702092def1545b21" Oct 30 15:58:14.130968 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.130880 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e497f068bb92fdfcb5f000d2e2e08ed0a37856789aa3ec3c702092def1545b21"} err="failed to get container status \"e497f068bb92fdfcb5f000d2e2e08ed0a37856789aa3ec3c702092def1545b21\": rpc error: code = NotFound desc = could not find container \"e497f068bb92fdfcb5f000d2e2e08ed0a37856789aa3ec3c702092def1545b21\": container with ID starting with e497f068bb92fdfcb5f000d2e2e08ed0a37856789aa3ec3c702092def1545b21 not found: ID does not exist" Oct 30 15:58:14.166768 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.150322 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8"] Oct 30 15:58:14.166768 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.163669 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-lk8t8"] Oct 30 15:58:14.203964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.203915 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996"] Oct 30 15:58:14.214426 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.214382 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-d4996"] Oct 30 15:58:14.215635 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.215570 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:14.253325 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.253294 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c"] Oct 30 15:58:14.257302 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.257272 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-hn78c"] Oct 30 15:58:14.634477 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.634446 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:14.833960 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:14.833914 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:15.272222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:15.271902 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:16.058706 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:16.058672 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0cb01099-f910-4f21-a0b6-ff5dbc33bf60" path="/var/lib/kubelet/pods/0cb01099-f910-4f21-a0b6-ff5dbc33bf60/volumes" Oct 30 15:58:16.059993 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:16.059969 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" path="/var/lib/kubelet/pods/46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92/volumes" Oct 30 15:58:16.060827 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:16.060807 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83bf7cb0-3eae-41b2-8ba0-4db7b4634071" path="/var/lib/kubelet/pods/83bf7cb0-3eae-41b2-8ba0-4db7b4634071/volumes" Oct 30 15:58:16.689413 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:16.689355 5013 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:58:16.977386 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:16.977284 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:58:17.046346 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:17.046306 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded" Oct 30 15:58:17.224226 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:17.222747 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded" Oct 30 15:58:17.527447 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:17.527409 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:19.431632 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:19.431588 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:20.158850 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:20.158822 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:21.338807 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:21.338776 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:22.339634 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:22.339591 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:23.197552 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:23.197518 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:23.568904 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:23.568802 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:23.733517 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:23.733487 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:24.587921 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:24.587888 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:24.986037 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:24.985999 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:25.529042 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:25.529011 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:26.617471 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:26.617438 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm"] Oct 30 15:58:26.617956 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:26.617697 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" podUID="8821e53a-f78f-4139-ab3a-1cbafb6fe5a7" containerName="maas-api" containerID="cri-o://a00627808f8d474eca32e1d5f4f435e41b4c174d53f90e9f51be7f8ff99ce751" gracePeriod=30 Oct 30 15:58:26.742114 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:26.742083 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:26.961095 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:26.961065 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:58:26.978958 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:26.978920 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-kube-api-access-lbsgm\" (UniqueName: \"kubernetes.io/projected/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-kube-api-access-lbsgm\") pod \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " Oct 30 15:58:26.979117 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:26.979065 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-maas-api-tls\") pod \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " Oct 30 15:58:26.979117 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:26.979099 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-metrics-certs\") pod \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " Oct 30 15:58:26.979283 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:26.979181 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-openshift-service-ca\") pod \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\" (UID: \"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7\") " Oct 30 15:58:26.980251 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:26.980129 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-openshift-service-ca" pod "8821e53a-f78f-4139-ab3a-1cbafb6fe5a7" (UID: "8821e53a-f78f-4139-ab3a-1cbafb6fe5a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:58:26.989364 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:26.989323 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-kube-api-access-lbsgm" pod "8821e53a-f78f-4139-ab3a-1cbafb6fe5a7" (UID: "8821e53a-f78f-4139-ab3a-1cbafb6fe5a7"). InnerVolumeSpecName "kube-api-access-lbsgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:26.989592 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:26.989565 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-metrics-certs" pod "8821e53a-f78f-4139-ab3a-1cbafb6fe5a7" (UID: "8821e53a-f78f-4139-ab3a-1cbafb6fe5a7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:58:26.989719 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:26.989633 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-maas-api-tls" pod "8821e53a-f78f-4139-ab3a-1cbafb6fe5a7" (UID: "8821e53a-f78f-4139-ab3a-1cbafb6fe5a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:58:27.030238 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:58:27.030156 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:58:27.081223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.080850 5013 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-maas-api-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:27.081223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.080883 5013 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-metrics-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:27.081223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.080898 5013 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-openshift-service-ca\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:27.081223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.080912 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbsgm\" (UniqueName: \"kubernetes.io/projected/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7-kube-api-access-lbsgm\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:27.299087 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.299005 5013 generic.go:356] "Generic (PLEG): container finished" podID="8821e53a-f78f-4139-ab3a-1cbafb6fe5a7" containerID="a00627808f8d474eca32e1d5f4f435e41b4c174d53f90e9f51be7f8ff99ce751" exitCode=0 Oct 30 15:58:27.299087 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.299052 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" event={"ID":"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7","Type":"ContainerDied","Data":"a00627808f8d474eca32e1d5f4f435e41b4c174d53f90e9f51be7f8ff99ce751"} Oct 30 15:58:27.299306 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.299089 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" event={"ID":"8821e53a-f78f-4139-ab3a-1cbafb6fe5a7","Type":"ContainerDied","Data":"30b55ad8336038e864b12a6c21ee9bd496773b13a58e8c5c23d0b4a3ecae954e"} Oct 30 15:58:27.299306 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.299109 5013 scope.go:122] "RemoveContainer" containerID="a00627808f8d474eca32e1d5f4f435e41b4c174d53f90e9f51be7f8ff99ce751" Oct 30 15:58:27.299306 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.299052 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm" Oct 30 15:58:27.314404 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.314380 5013 scope.go:122] "RemoveContainer" containerID="a00627808f8d474eca32e1d5f4f435e41b4c174d53f90e9f51be7f8ff99ce751" Oct 30 15:58:27.314539 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.314503 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:27.314763 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:58:27.314744 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a00627808f8d474eca32e1d5f4f435e41b4c174d53f90e9f51be7f8ff99ce751\": container with ID starting with a00627808f8d474eca32e1d5f4f435e41b4c174d53f90e9f51be7f8ff99ce751 not found: ID does not exist" containerID="a00627808f8d474eca32e1d5f4f435e41b4c174d53f90e9f51be7f8ff99ce751" Oct 30 15:58:27.314814 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.314773 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a00627808f8d474eca32e1d5f4f435e41b4c174d53f90e9f51be7f8ff99ce751"} err="failed to get container status \"a00627808f8d474eca32e1d5f4f435e41b4c174d53f90e9f51be7f8ff99ce751\": rpc error: code = NotFound desc = could not find container \"a00627808f8d474eca32e1d5f4f435e41b4c174d53f90e9f51be7f8ff99ce751\": container with ID starting with a00627808f8d474eca32e1d5f4f435e41b4c174d53f90e9f51be7f8ff99ce751 not found: ID does not exist" Oct 30 15:58:27.349163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.349135 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm"] Oct 30 15:58:27.364079 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.364039 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b24e7b-555b8f5fdb-8qjvm"] Oct 30 15:58:27.482757 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.482713 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:27.945773 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:27.945735 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:28.006827 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:28.006797 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8821e53a-f78f-4139-ab3a-1cbafb6fe5a7" path="/var/lib/kubelet/pods/8821e53a-f78f-4139-ab3a-1cbafb6fe5a7/volumes" Oct 30 15:58:28.651986 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:28.651952 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:29.469079 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:29.469045 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:31.114186 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:31.114149 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:31.381772 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:31.381746 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:33.106687 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:33.106650 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:34.763510 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:34.763468 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:35.079836 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:35.079728 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:35.258909 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:35.258881 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:35.516651 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:35.516619 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:35.549017 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:35.548977 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq"] Oct 30 15:58:35.549272 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:35.549251 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" podUID="1b325488-f2d5-49c8-9c76-d70554d9f0fe" containerName="istio-proxy" containerID="cri-o://35e78a306d5627a5a97aac734bf36666209444de7ac21fe33d7538f9a9a6c3cd" gracePeriod=30 Oct 30 15:58:35.551518 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:35.551500 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr"] Oct 30 15:58:35.551693 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:35.551661 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" podUID="f0b22fc2-cafd-4d44-ab78-d1020db061fa" containerName="istio-proxy" containerID="cri-o://f3e72b8621810f24e8b164e56431e1530ab55df8ad5a2ee8cd5865d2b771bbe7" gracePeriod=30 Oct 30 15:58:35.560241 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:35.560187 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz"] Oct 30 15:58:35.560433 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:35.560403 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" podUID="c0fda42f-3851-4542-a220-b9591e30d2a7" containerName="istio-proxy" containerID="cri-o://2b3bd4ec04774e632314ac6f294d13dcad563bdaa43d58044f7a03f24d235dd4" gracePeriod=30 Oct 30 15:58:35.826252 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:35.826149 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 15:58:35.826252 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:35.826218 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" podUID="c0fda42f-3851-4542-a220-b9591e30d2a7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 15:58:35.882186 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:35.882149 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 15:58:35.882368 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:35.882228 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" podUID="1b325488-f2d5-49c8-9c76-d70554d9f0fe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 15:58:36.226914 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:36.226876 5013 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 15:58:36.227113 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:36.226938 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" podUID="f0b22fc2-cafd-4d44-ab78-d1020db061fa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 15:58:36.766297 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:36.766267 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:38.281734 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:38.281701 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:38.449293 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:38.449263 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:38.619304 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:38.619227 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:41.024586 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.023940 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 15:58:41.373347 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.370167 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:58:41.649291 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.648024 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0b22fc2-cafd-4d44-ab78-d1020db061fa-kube-api-access-8x9f4\" (UniqueName: \"kubernetes.io/projected/f0b22fc2-cafd-4d44-ab78-d1020db061fa-kube-api-access-8x9f4\") pod \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " Oct 30 15:58:41.649291 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.648107 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-workload-certs\") pod \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " Oct 30 15:58:41.649291 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.648160 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-credential-socket\") pod \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " Oct 30 15:58:41.649291 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.648274 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-podinfo\") pod \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " Oct 30 15:58:41.649291 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.648316 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-token\" (UniqueName: \"kubernetes.io/projected/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-token\") pod \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " Oct 30 15:58:41.649291 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.648401 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istiod-ca-cert\") pod \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " Oct 30 15:58:41.650286 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.649895 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istiod-ca-cert" pod "f0b22fc2-cafd-4d44-ab78-d1020db061fa" (UID: "f0b22fc2-cafd-4d44-ab78-d1020db061fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:58:41.650747 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.650690 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-credential-socket" pod "f0b22fc2-cafd-4d44-ab78-d1020db061fa" (UID: "f0b22fc2-cafd-4d44-ab78-d1020db061fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:41.653678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.653376 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-workload-certs" pod "f0b22fc2-cafd-4d44-ab78-d1020db061fa" (UID: "f0b22fc2-cafd-4d44-ab78-d1020db061fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:41.723222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.690271 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0b22fc2-cafd-4d44-ab78-d1020db061fa-kube-api-access-8x9f4" pod "f0b22fc2-cafd-4d44-ab78-d1020db061fa" (UID: "f0b22fc2-cafd-4d44-ab78-d1020db061fa"). InnerVolumeSpecName "kube-api-access-8x9f4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:41.723222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.697135 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-podinfo" pod "f0b22fc2-cafd-4d44-ab78-d1020db061fa" (UID: "f0b22fc2-cafd-4d44-ab78-d1020db061fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:58:41.723222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.709613 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:41.791691 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.765377 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-envoy\") pod \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " Oct 30 15:58:41.817221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.793346 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-data\") pod \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " Oct 30 15:58:41.817221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.793402 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-workload-socket\") pod \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\" (UID: \"f0b22fc2-cafd-4d44-ab78-d1020db061fa\") " Oct 30 15:58:41.817221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.794324 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:41.817221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.794355 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:41.817221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.794371 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x9f4\" (UniqueName: \"kubernetes.io/projected/f0b22fc2-cafd-4d44-ab78-d1020db061fa-kube-api-access-8x9f4\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:41.817221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.794386 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:41.817221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.794406 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:41.817221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.766919 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-token" pod "f0b22fc2-cafd-4d44-ab78-d1020db061fa" (UID: "f0b22fc2-cafd-4d44-ab78-d1020db061fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:41.817221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.784288 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-envoy" pod "f0b22fc2-cafd-4d44-ab78-d1020db061fa" (UID: "f0b22fc2-cafd-4d44-ab78-d1020db061fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:41.817221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.805282 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-workload-socket" pod "f0b22fc2-cafd-4d44-ab78-d1020db061fa" (UID: "f0b22fc2-cafd-4d44-ab78-d1020db061fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:41.817221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.805325 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-data" pod "f0b22fc2-cafd-4d44-ab78-d1020db061fa" (UID: "f0b22fc2-cafd-4d44-ab78-d1020db061fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:41.992800 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.968176 5013 generic.go:356] "Generic (PLEG): container finished" podID="c0fda42f-3851-4542-a220-b9591e30d2a7" containerID="2b3bd4ec04774e632314ac6f294d13dcad563bdaa43d58044f7a03f24d235dd4" exitCode=0 Oct 30 15:58:41.992800 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.968442 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" event={"ID":"c0fda42f-3851-4542-a220-b9591e30d2a7","Type":"ContainerDied","Data":"2b3bd4ec04774e632314ac6f294d13dcad563bdaa43d58044f7a03f24d235dd4"} Oct 30 15:58:42.011411 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:41.994871 5013 generic.go:356] "Generic (PLEG): container finished" podID="1b325488-f2d5-49c8-9c76-d70554d9f0fe" containerID="35e78a306d5627a5a97aac734bf36666209444de7ac21fe33d7538f9a9a6c3cd" exitCode=0 Oct 30 15:58:42.011411 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.009056 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" event={"ID":"1b325488-f2d5-49c8-9c76-d70554d9f0fe","Type":"ContainerDied","Data":"35e78a306d5627a5a97aac734bf36666209444de7ac21fe33d7538f9a9a6c3cd"} Oct 30 15:58:42.020624 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.012038 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.020624 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.012103 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.020624 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.012120 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.020624 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.012143 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0b22fc2-cafd-4d44-ab78-d1020db061fa-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.056936 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.056171 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 15:58:42.133216 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.129748 5013 generic.go:356] "Generic (PLEG): container finished" podID="f0b22fc2-cafd-4d44-ab78-d1020db061fa" containerID="f3e72b8621810f24e8b164e56431e1530ab55df8ad5a2ee8cd5865d2b771bbe7" exitCode=0 Oct 30 15:58:42.133216 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.129867 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" Oct 30 15:58:42.135150 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.135114 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:58:42.149229 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.146458 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:58:42.149229 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.146502 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" event={"ID":"f0b22fc2-cafd-4d44-ab78-d1020db061fa","Type":"ContainerDied","Data":"f3e72b8621810f24e8b164e56431e1530ab55df8ad5a2ee8cd5865d2b771bbe7"} Oct 30 15:58:42.149229 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.146604 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr" event={"ID":"f0b22fc2-cafd-4d44-ab78-d1020db061fa","Type":"ContainerDied","Data":"db2ea4fd9dd37d2b2157b506be6bd9335385f5fa8164a80de0bcaca2d74799f7"} Oct 30 15:58:42.153297 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.153257 5013 scope.go:122] "RemoveContainer" containerID="f3e72b8621810f24e8b164e56431e1530ab55df8ad5a2ee8cd5865d2b771bbe7" Oct 30 15:58:42.194974 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.194880 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:58:42.197878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.197818 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:58:42.229929 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.229875 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istiod-ca-cert\") pod \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " Oct 30 15:58:42.230267 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.230227 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-credential-socket\") pod \"c0fda42f-3851-4542-a220-b9591e30d2a7\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " Oct 30 15:58:42.230516 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.230473 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b325488-f2d5-49c8-9c76-d70554d9f0fe-kube-api-access-4zdww\" (UniqueName: \"kubernetes.io/projected/1b325488-f2d5-49c8-9c76-d70554d9f0fe-kube-api-access-4zdww\") pod \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " Oct 30 15:58:42.230718 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.230700 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-workload-socket\") pod \"c0fda42f-3851-4542-a220-b9591e30d2a7\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " Oct 30 15:58:42.230918 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.230874 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-data\") pod \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " Oct 30 15:58:42.231108 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.231064 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-data\") pod \"c0fda42f-3851-4542-a220-b9591e30d2a7\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " Oct 30 15:58:42.231325 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.231305 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-token\" (UniqueName: \"kubernetes.io/projected/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-token\") pod \"c0fda42f-3851-4542-a220-b9591e30d2a7\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " Oct 30 15:58:42.231709 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.231673 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-token\" (UniqueName: \"kubernetes.io/projected/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-token\") pod \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " Oct 30 15:58:42.232325 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.232301 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-podinfo\") pod \"c0fda42f-3851-4542-a220-b9591e30d2a7\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " Oct 30 15:58:42.232518 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.232488 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-envoy\") pod \"c0fda42f-3851-4542-a220-b9591e30d2a7\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " Oct 30 15:58:42.232678 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.232660 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-workload-socket\") pod \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " Oct 30 15:58:42.232842 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.232825 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0fda42f-3851-4542-a220-b9591e30d2a7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0fda42f-3851-4542-a220-b9591e30d2a7-istiod-ca-cert\") pod \"c0fda42f-3851-4542-a220-b9591e30d2a7\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " Oct 30 15:58:42.233013 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.232984 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0fda42f-3851-4542-a220-b9591e30d2a7-kube-api-access-z4t67\" (UniqueName: \"kubernetes.io/projected/c0fda42f-3851-4542-a220-b9591e30d2a7-kube-api-access-z4t67\") pod \"c0fda42f-3851-4542-a220-b9591e30d2a7\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " Oct 30 15:58:42.234162 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.233272 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-credential-socket\") pod \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " Oct 30 15:58:42.236624 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.236590 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-workload-certs\") pod \"c0fda42f-3851-4542-a220-b9591e30d2a7\" (UID: \"c0fda42f-3851-4542-a220-b9591e30d2a7\") " Oct 30 15:58:42.236835 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.236807 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-podinfo\") pod \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " Oct 30 15:58:42.237014 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.236986 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-workload-certs\") pod \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " Oct 30 15:58:42.237169 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.237152 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-envoy\") pod \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\" (UID: \"1b325488-f2d5-49c8-9c76-d70554d9f0fe\") " Oct 30 15:58:42.237292 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.237260 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istiod-ca-cert" pod "1b325488-f2d5-49c8-9c76-d70554d9f0fe" (UID: "1b325488-f2d5-49c8-9c76-d70554d9f0fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:58:42.237698 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.237677 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-data" pod "1b325488-f2d5-49c8-9c76-d70554d9f0fe" (UID: "1b325488-f2d5-49c8-9c76-d70554d9f0fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:42.239137 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.239113 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-workload-socket" pod "c0fda42f-3851-4542-a220-b9591e30d2a7" (UID: "c0fda42f-3851-4542-a220-b9591e30d2a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:42.239469 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.239449 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-credential-socket" pod "c0fda42f-3851-4542-a220-b9591e30d2a7" (UID: "c0fda42f-3851-4542-a220-b9591e30d2a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:42.239753 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.239733 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-data" pod "c0fda42f-3851-4542-a220-b9591e30d2a7" (UID: "c0fda42f-3851-4542-a220-b9591e30d2a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:42.242988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.240956 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-workload-certs" pod "1b325488-f2d5-49c8-9c76-d70554d9f0fe" (UID: "1b325488-f2d5-49c8-9c76-d70554d9f0fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:42.242988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.241179 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-credential-socket" pod "1b325488-f2d5-49c8-9c76-d70554d9f0fe" (UID: "1b325488-f2d5-49c8-9c76-d70554d9f0fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:42.242988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.241380 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-workload-certs" pod "c0fda42f-3851-4542-a220-b9591e30d2a7" (UID: "c0fda42f-3851-4542-a220-b9591e30d2a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:42.242988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.241540 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.242988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.241561 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.242988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.241575 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.242988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.241590 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.242988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.241602 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.242988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.241614 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.242988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.241627 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.242988 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.241641 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.243926 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.243906 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-workload-socket" pod "1b325488-f2d5-49c8-9c76-d70554d9f0fe" (UID: "1b325488-f2d5-49c8-9c76-d70554d9f0fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:42.244369 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.244349 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0fda42f-3851-4542-a220-b9591e30d2a7-istiod-ca-cert" pod "c0fda42f-3851-4542-a220-b9591e30d2a7" (UID: "c0fda42f-3851-4542-a220-b9591e30d2a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:58:42.247852 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.247829 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-envoy" pod "1b325488-f2d5-49c8-9c76-d70554d9f0fe" (UID: "1b325488-f2d5-49c8-9c76-d70554d9f0fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:42.249366 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.249331 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-envoy" pod "c0fda42f-3851-4542-a220-b9591e30d2a7" (UID: "c0fda42f-3851-4542-a220-b9591e30d2a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:58:42.250018 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.249983 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-token" pod "1b325488-f2d5-49c8-9c76-d70554d9f0fe" (UID: "1b325488-f2d5-49c8-9c76-d70554d9f0fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:42.250309 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.250267 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-podinfo" pod "c0fda42f-3851-4542-a220-b9591e30d2a7" (UID: "c0fda42f-3851-4542-a220-b9591e30d2a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:58:42.256114 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.255828 5013 scope.go:122] "RemoveContainer" containerID="f3e72b8621810f24e8b164e56431e1530ab55df8ad5a2ee8cd5865d2b771bbe7" Oct 30 15:58:42.256114 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.255964 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-podinfo" pod "1b325488-f2d5-49c8-9c76-d70554d9f0fe" (UID: "1b325488-f2d5-49c8-9c76-d70554d9f0fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:58:42.256114 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.256050 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0fda42f-3851-4542-a220-b9591e30d2a7-kube-api-access-z4t67" pod "c0fda42f-3851-4542-a220-b9591e30d2a7" (UID: "c0fda42f-3851-4542-a220-b9591e30d2a7"). InnerVolumeSpecName "kube-api-access-z4t67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:42.256782 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:58:42.256698 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3e72b8621810f24e8b164e56431e1530ab55df8ad5a2ee8cd5865d2b771bbe7\": container with ID starting with f3e72b8621810f24e8b164e56431e1530ab55df8ad5a2ee8cd5865d2b771bbe7 not found: ID does not exist" containerID="f3e72b8621810f24e8b164e56431e1530ab55df8ad5a2ee8cd5865d2b771bbe7" Oct 30 15:58:42.256782 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.256737 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3e72b8621810f24e8b164e56431e1530ab55df8ad5a2ee8cd5865d2b771bbe7"} err="failed to get container status \"f3e72b8621810f24e8b164e56431e1530ab55df8ad5a2ee8cd5865d2b771bbe7\": rpc error: code = NotFound desc = could not find container \"f3e72b8621810f24e8b164e56431e1530ab55df8ad5a2ee8cd5865d2b771bbe7\": container with ID starting with f3e72b8621810f24e8b164e56431e1530ab55df8ad5a2ee8cd5865d2b771bbe7 not found: ID does not exist" Oct 30 15:58:42.257378 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.257341 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b325488-f2d5-49c8-9c76-d70554d9f0fe-kube-api-access-4zdww" pod "1b325488-f2d5-49c8-9c76-d70554d9f0fe" (UID: "1b325488-f2d5-49c8-9c76-d70554d9f0fe"). InnerVolumeSpecName "kube-api-access-4zdww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:42.257618 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.257589 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-token" pod "c0fda42f-3851-4542-a220-b9591e30d2a7" (UID: "c0fda42f-3851-4542-a220-b9591e30d2a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:58:42.267277 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.266922 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:42.349144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.349015 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.349144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.349047 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.349144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.349057 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.349144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.349065 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.349144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.349073 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0fda42f-3851-4542-a220-b9591e30d2a7-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.349144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.349082 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4t67\" (UniqueName: \"kubernetes.io/projected/c0fda42f-3851-4542-a220-b9591e30d2a7-kube-api-access-z4t67\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.349144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.349091 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.349144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.349100 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b325488-f2d5-49c8-9c76-d70554d9f0fe-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.349144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.349110 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zdww\" (UniqueName: \"kubernetes.io/projected/1b325488-f2d5-49c8-9c76-d70554d9f0fe-kube-api-access-4zdww\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.349144 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.349123 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0fda42f-3851-4542-a220-b9591e30d2a7-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:58:42.385693 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.385623 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr"] Oct 30 15:58:42.385851 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:42.385821 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-28ttr"] Oct 30 15:58:43.138822 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:43.138791 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" Oct 30 15:58:43.138822 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:43.138803 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz" event={"ID":"c0fda42f-3851-4542-a220-b9591e30d2a7","Type":"ContainerDied","Data":"3ab09b82996288641c57070d17e75dc887987fbecbb82a4b9318f82cd982c325"} Oct 30 15:58:43.139372 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:43.138871 5013 scope.go:122] "RemoveContainer" containerID="2b3bd4ec04774e632314ac6f294d13dcad563bdaa43d58044f7a03f24d235dd4" Oct 30 15:58:43.140757 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:43.140497 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" Oct 30 15:58:43.140757 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:43.140525 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq" event={"ID":"1b325488-f2d5-49c8-9c76-d70554d9f0fe","Type":"ContainerDied","Data":"809644c0e2f49271e8d9aec587c31fd3028fb2995908d51ebae3d6b0270cd85a"} Oct 30 15:58:43.152467 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:43.152274 5013 scope.go:122] "RemoveContainer" containerID="35e78a306d5627a5a97aac734bf36666209444de7ac21fe33d7538f9a9a6c3cd" Oct 30 15:58:43.193973 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:43.193948 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz"] Oct 30 15:58:43.207571 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:43.207542 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-q79jz"] Oct 30 15:58:43.241037 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:43.241001 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq"] Oct 30 15:58:43.259190 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:43.259159 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-57cjq"] Oct 30 15:58:44.006884 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:44.006848 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b325488-f2d5-49c8-9c76-d70554d9f0fe" path="/var/lib/kubelet/pods/1b325488-f2d5-49c8-9c76-d70554d9f0fe/volumes" Oct 30 15:58:44.007465 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:44.007443 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0fda42f-3851-4542-a220-b9591e30d2a7" path="/var/lib/kubelet/pods/c0fda42f-3851-4542-a220-b9591e30d2a7/volumes" Oct 30 15:58:44.007919 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:44.007905 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0b22fc2-cafd-4d44-ab78-d1020db061fa" path="/var/lib/kubelet/pods/f0b22fc2-cafd-4d44-ab78-d1020db061fa/volumes" Oct 30 15:58:44.142538 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:44.142505 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:44.310593 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:44.310521 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:44.550472 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:44.550442 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:46.319245 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:46.319167 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:46.550742 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:46.550707 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n"] Oct 30 15:58:46.719641 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:46.719611 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:48.290409 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:48.290378 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:49.033143 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:49.033110 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29"] Oct 30 15:58:49.046381 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:49.046351 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f"] Oct 30 15:58:49.197908 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:49.197879 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:49.595859 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:49.595830 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:49.958508 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:49.958476 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:51.866775 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:51.866745 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:52.551361 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:52.551327 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:52.723341 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:52.723310 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:53.629961 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:53.627508 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:54.888142 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:54.888113 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:55.052371 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:55.052339 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:55.470973 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:55.470928 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:55.633379 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:55.633339 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:55.801024 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:55.800941 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:56.977440 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:58:56.977401 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:58:57.645098 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:57.645050 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:58:59.354701 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:58:59.354667 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:01.188014 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.187977 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:01.572538 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.572437 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" containerID="cri-o://76d3bd542e85243573a8287f44449eade4b06560050231f2a5c20bd4469ed877" gracePeriod=45 Oct 30 15:59:01.842918 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.842889 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:59:01.986603 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.986574 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd53a00a-b214-47dd-8a64-af52d0e490a5-kube-api-access-vjxq2\" (UniqueName: \"kubernetes.io/projected/dd53a00a-b214-47dd-8a64-af52d0e490a5-kube-api-access-vjxq2\") pod \"dd53a00a-b214-47dd-8a64-af52d0e490a5\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " Oct 30 15:59:01.986786 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.986636 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-model-cache\") pod \"dd53a00a-b214-47dd-8a64-af52d0e490a5\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " Oct 30 15:59:01.986786 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.986666 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dd53a00a-b214-47dd-8a64-af52d0e490a5-tls-certs\" (UniqueName: \"kubernetes.io/secret/dd53a00a-b214-47dd-8a64-af52d0e490a5-tls-certs\") pod \"dd53a00a-b214-47dd-8a64-af52d0e490a5\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " Oct 30 15:59:01.986786 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.986691 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-home\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-home\") pod \"dd53a00a-b214-47dd-8a64-af52d0e490a5\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " Oct 30 15:59:01.986786 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.986748 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-tmp-dir\") pod \"dd53a00a-b214-47dd-8a64-af52d0e490a5\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " Oct 30 15:59:01.987008 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.986819 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-dshm\") pod \"dd53a00a-b214-47dd-8a64-af52d0e490a5\" (UID: \"dd53a00a-b214-47dd-8a64-af52d0e490a5\") " Oct 30 15:59:01.987008 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.986956 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-model-cache" pod "dd53a00a-b214-47dd-8a64-af52d0e490a5" (UID: "dd53a00a-b214-47dd-8a64-af52d0e490a5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:01.987118 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.986998 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-home" pod "dd53a00a-b214-47dd-8a64-af52d0e490a5" (UID: "dd53a00a-b214-47dd-8a64-af52d0e490a5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:01.987118 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.987084 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-tmp-dir" pod "dd53a00a-b214-47dd-8a64-af52d0e490a5" (UID: "dd53a00a-b214-47dd-8a64-af52d0e490a5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:01.987327 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.987313 5013 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-home\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:01.987380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.987329 5013 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-tmp-dir\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:01.987380 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.987338 5013 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-model-cache\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:01.989158 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.989138 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-dshm" pod "dd53a00a-b214-47dd-8a64-af52d0e490a5" (UID: "dd53a00a-b214-47dd-8a64-af52d0e490a5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:01.989396 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.989383 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd53a00a-b214-47dd-8a64-af52d0e490a5-tls-certs" pod "dd53a00a-b214-47dd-8a64-af52d0e490a5" (UID: "dd53a00a-b214-47dd-8a64-af52d0e490a5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:59:01.989449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:01.989405 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd53a00a-b214-47dd-8a64-af52d0e490a5-kube-api-access-vjxq2" pod "dd53a00a-b214-47dd-8a64-af52d0e490a5" (UID: "dd53a00a-b214-47dd-8a64-af52d0e490a5"). InnerVolumeSpecName "kube-api-access-vjxq2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:02.088650 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:02.088571 5013 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd53a00a-b214-47dd-8a64-af52d0e490a5-dshm\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:02.088650 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:02.088600 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjxq2\" (UniqueName: \"kubernetes.io/projected/dd53a00a-b214-47dd-8a64-af52d0e490a5-kube-api-access-vjxq2\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:02.088650 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:02.088613 5013 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd53a00a-b214-47dd-8a64-af52d0e490a5-tls-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:02.260546 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:02.260507 5013 generic.go:356] "Generic (PLEG): container finished" podID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerID="76d3bd542e85243573a8287f44449eade4b06560050231f2a5c20bd4469ed877" exitCode=0 Oct 30 15:59:02.260951 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:02.260567 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" Oct 30 15:59:02.260951 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:02.260575 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" event={"ID":"dd53a00a-b214-47dd-8a64-af52d0e490a5","Type":"ContainerDied","Data":"76d3bd542e85243573a8287f44449eade4b06560050231f2a5c20bd4469ed877"} Oct 30 15:59:02.260951 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:02.260625 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" event={"ID":"dd53a00a-b214-47dd-8a64-af52d0e490a5","Type":"ContainerDied","Data":"b53a0e2e90b3c4bcd14d7d0ce64e42de48d2f2f4a156c9f5fa87786ae89d40b8"} Oct 30 15:59:02.260951 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:02.260648 5013 scope.go:122] "RemoveContainer" containerID="76d3bd542e85243573a8287f44449eade4b06560050231f2a5c20bd4469ed877" Oct 30 15:59:02.270350 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:02.270332 5013 scope.go:122] "RemoveContainer" containerID="76d3bd542e85243573a8287f44449eade4b06560050231f2a5c20bd4469ed877" Oct 30 15:59:02.270607 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:59:02.270586 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76d3bd542e85243573a8287f44449eade4b06560050231f2a5c20bd4469ed877\": container with ID starting with 76d3bd542e85243573a8287f44449eade4b06560050231f2a5c20bd4469ed877 not found: ID does not exist" containerID="76d3bd542e85243573a8287f44449eade4b06560050231f2a5c20bd4469ed877" Oct 30 15:59:02.270657 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:02.270615 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76d3bd542e85243573a8287f44449eade4b06560050231f2a5c20bd4469ed877"} err="failed to get container status \"76d3bd542e85243573a8287f44449eade4b06560050231f2a5c20bd4469ed877\": rpc error: code = NotFound desc = could not find container \"76d3bd542e85243573a8287f44449eade4b06560050231f2a5c20bd4469ed877\": container with ID starting with 76d3bd542e85243573a8287f44449eade4b06560050231f2a5c20bd4469ed877 not found: ID does not exist" Oct 30 15:59:02.311628 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:02.311605 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n"] Oct 30 15:59:02.322956 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:02.322931 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n"] Oct 30 15:59:02.751118 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:02.751076 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/premium-w3-040f9a-kserve-5997dfdf79-vzk2n" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 15:59:03.112400 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:03.112319 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:04.005808 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.005777 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" path="/var/lib/kubelet/pods/dd53a00a-b214-47dd-8a64-af52d0e490a5/volumes" Oct 30 15:59:04.057805 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.057754 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerName="main" containerID="cri-o://918c46e9d894d3330620e719f08439162581a0ba02668482f4810db3c39baefc" gracePeriod=45 Oct 30 15:59:04.067659 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.067619 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" containerID="cri-o://6b327307c5e16e4f4124a4c68a635f2a0ea3da6aeeabe720cd0f3c70befa9c46" gracePeriod=45 Oct 30 15:59:04.182440 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.182409 5013 scope.go:122] "RemoveContainer" containerID="e034790e8f537c1b3e443c50433aa934d1df5b7c06c5052d44a1de26423c0e7e" Oct 30 15:59:04.198371 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.198339 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": dial tcp 10.132.0.73:8000: connect: connection refused" Oct 30 15:59:04.277148 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.275620 5013 generic.go:356] "Generic (PLEG): container finished" podID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerID="6b327307c5e16e4f4124a4c68a635f2a0ea3da6aeeabe720cd0f3c70befa9c46" exitCode=0 Oct 30 15:59:04.277148 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.275807 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" event={"ID":"82fbbdd5-fc1a-421a-9bc2-865304cac190","Type":"ContainerDied","Data":"6b327307c5e16e4f4124a4c68a635f2a0ea3da6aeeabe720cd0f3c70befa9c46"} Oct 30 15:59:04.277738 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.277706 5013 generic.go:356] "Generic (PLEG): container finished" podID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerID="918c46e9d894d3330620e719f08439162581a0ba02668482f4810db3c39baefc" exitCode=0 Oct 30 15:59:04.277834 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.277761 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" event={"ID":"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba","Type":"ContainerDied","Data":"918c46e9d894d3330620e719f08439162581a0ba02668482f4810db3c39baefc"} Oct 30 15:59:04.345864 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.345837 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:59:04.392335 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.392315 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:59:04.515366 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515285 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-dshm\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-dshm\") pod \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " Oct 30 15:59:04.515366 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515322 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-tmp-dir\") pod \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " Oct 30 15:59:04.515366 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515347 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-model-cache\") pod \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " Oct 30 15:59:04.515643 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515411 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-dshm\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-dshm\") pod \"82fbbdd5-fc1a-421a-9bc2-865304cac190\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " Oct 30 15:59:04.515643 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515506 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82fbbdd5-fc1a-421a-9bc2-865304cac190-kube-api-access-x4lzm\" (UniqueName: \"kubernetes.io/projected/82fbbdd5-fc1a-421a-9bc2-865304cac190-kube-api-access-x4lzm\") pod \"82fbbdd5-fc1a-421a-9bc2-865304cac190\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " Oct 30 15:59:04.515643 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515535 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-kube-api-access-x77gx\" (UniqueName: \"kubernetes.io/projected/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-kube-api-access-x77gx\") pod \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " Oct 30 15:59:04.515643 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515571 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/82fbbdd5-fc1a-421a-9bc2-865304cac190-tls-certs\" (UniqueName: \"kubernetes.io/secret/82fbbdd5-fc1a-421a-9bc2-865304cac190-tls-certs\") pod \"82fbbdd5-fc1a-421a-9bc2-865304cac190\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " Oct 30 15:59:04.515643 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515571 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-tmp-dir" pod "af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" (UID: "af0c6ebb-6ae6-455e-9b35-e72ffb5538ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:04.515643 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515607 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-model-cache" pod "af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" (UID: "af0c6ebb-6ae6-455e-9b35-e72ffb5538ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:04.515958 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515654 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-tmp-dir\") pod \"82fbbdd5-fc1a-421a-9bc2-865304cac190\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " Oct 30 15:59:04.515958 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515705 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-home\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-home\") pod \"82fbbdd5-fc1a-421a-9bc2-865304cac190\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " Oct 30 15:59:04.515958 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515748 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-tls-certs\" (UniqueName: \"kubernetes.io/secret/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-tls-certs\") pod \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " Oct 30 15:59:04.515958 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515774 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-home\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-home\") pod \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\" (UID: \"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba\") " Oct 30 15:59:04.515958 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515858 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-model-cache\") pod \"82fbbdd5-fc1a-421a-9bc2-865304cac190\" (UID: \"82fbbdd5-fc1a-421a-9bc2-865304cac190\") " Oct 30 15:59:04.516242 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515964 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-home" pod "82fbbdd5-fc1a-421a-9bc2-865304cac190" (UID: "82fbbdd5-fc1a-421a-9bc2-865304cac190"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:04.516242 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.515977 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-tmp-dir" pod "82fbbdd5-fc1a-421a-9bc2-865304cac190" (UID: "82fbbdd5-fc1a-421a-9bc2-865304cac190"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:04.516535 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.516385 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-home" pod "af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" (UID: "af0c6ebb-6ae6-455e-9b35-e72ffb5538ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:04.516535 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.516405 5013 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-tmp-dir\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:04.516535 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.516425 5013 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-home\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:04.516535 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.516439 5013 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-tmp-dir\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:04.516535 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.516453 5013 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-model-cache\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:04.516803 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.516640 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-model-cache" pod "82fbbdd5-fc1a-421a-9bc2-865304cac190" (UID: "82fbbdd5-fc1a-421a-9bc2-865304cac190"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:04.518048 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.518019 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82fbbdd5-fc1a-421a-9bc2-865304cac190-kube-api-access-x4lzm" pod "82fbbdd5-fc1a-421a-9bc2-865304cac190" (UID: "82fbbdd5-fc1a-421a-9bc2-865304cac190"). InnerVolumeSpecName "kube-api-access-x4lzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:04.518185 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.518137 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82fbbdd5-fc1a-421a-9bc2-865304cac190-tls-certs" pod "82fbbdd5-fc1a-421a-9bc2-865304cac190" (UID: "82fbbdd5-fc1a-421a-9bc2-865304cac190"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:59:04.518298 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.518269 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-dshm" pod "82fbbdd5-fc1a-421a-9bc2-865304cac190" (UID: "82fbbdd5-fc1a-421a-9bc2-865304cac190"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:04.518298 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.518279 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-kube-api-access-x77gx" pod "af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" (UID: "af0c6ebb-6ae6-455e-9b35-e72ffb5538ba"). InnerVolumeSpecName "kube-api-access-x77gx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:04.518725 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.518704 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-dshm" pod "af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" (UID: "af0c6ebb-6ae6-455e-9b35-e72ffb5538ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:04.519028 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.519015 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-tls-certs" pod "af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" (UID: "af0c6ebb-6ae6-455e-9b35-e72ffb5538ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:59:04.617343 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.617318 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4lzm\" (UniqueName: \"kubernetes.io/projected/82fbbdd5-fc1a-421a-9bc2-865304cac190-kube-api-access-x4lzm\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:04.617343 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.617340 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x77gx\" (UniqueName: \"kubernetes.io/projected/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-kube-api-access-x77gx\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:04.617343 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.617350 5013 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82fbbdd5-fc1a-421a-9bc2-865304cac190-tls-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:04.617571 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.617361 5013 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-tls-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:04.617571 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.617369 5013 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-home\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:04.617571 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.617377 5013 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-model-cache\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:04.617571 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.617384 5013 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba-dshm\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:04.617571 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.617391 5013 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82fbbdd5-fc1a-421a-9bc2-865304cac190-dshm\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:04.746732 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:04.746682 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:05.286290 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:05.286262 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" Oct 30 15:59:05.286774 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:05.286295 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29" event={"ID":"af0c6ebb-6ae6-455e-9b35-e72ffb5538ba","Type":"ContainerDied","Data":"99b4c46cb9747cc5aee9cc06907bd1d5e05e3a98433e583b2750161754dbb7ac"} Oct 30 15:59:05.286774 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:05.286344 5013 scope.go:122] "RemoveContainer" containerID="918c46e9d894d3330620e719f08439162581a0ba02668482f4810db3c39baefc" Oct 30 15:59:05.288054 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:05.288029 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" event={"ID":"82fbbdd5-fc1a-421a-9bc2-865304cac190","Type":"ContainerDied","Data":"a5aecf3c795ee410b3e62bab2b26f7a17da886155084f47a0dce679cc628e2cc"} Oct 30 15:59:05.288054 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:05.288053 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f" Oct 30 15:59:05.297804 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:05.297789 5013 scope.go:122] "RemoveContainer" containerID="6b327307c5e16e4f4124a4c68a635f2a0ea3da6aeeabe720cd0f3c70befa9c46" Oct 30 15:59:05.339070 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:05.339010 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f"] Oct 30 15:59:05.350607 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:05.350581 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-a2b3f1/premium-w1-a2b3f1-kserve-9d48bd76b-vqz6f"] Oct 30 15:59:05.382689 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:05.382648 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29"] Oct 30 15:59:05.394981 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:05.394948 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-a2b3f1/distinct-w1-a2b3f1-kserve-7db6b78fb9-8tq29"] Oct 30 15:59:06.016781 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:06.016746 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" path="/var/lib/kubelet/pods/82fbbdd5-fc1a-421a-9bc2-865304cac190/volumes" Oct 30 15:59:06.017274 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:06.017255 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" path="/var/lib/kubelet/pods/af0c6ebb-6ae6-455e-9b35-e72ffb5538ba/volumes" Oct 30 15:59:06.893848 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:06.893816 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:08.730642 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:08.730613 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:10.558403 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:10.558365 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:12.236160 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.236122 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:12.561033 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.560954 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z"] Oct 30 15:59:12.561452 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561439 5013 cpu_manager.go:414] "containerMap: removing container" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" Oct 30 15:59:12.561518 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561453 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" Oct 30 15:59:12.561518 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561465 5013 cpu_manager.go:414] "containerMap: removing container" podUID="83bf7cb0-3eae-41b2-8ba0-4db7b4634071" containerName="istio-proxy" Oct 30 15:59:12.561518 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561470 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83bf7cb0-3eae-41b2-8ba0-4db7b4634071" containerName="istio-proxy" Oct 30 15:59:12.561518 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561477 5013 cpu_manager.go:414] "containerMap: removing container" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerName="main" Oct 30 15:59:12.561518 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561482 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerName="main" Oct 30 15:59:12.561518 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561496 5013 cpu_manager.go:414] "containerMap: removing container" podUID="c0fda42f-3851-4542-a220-b9591e30d2a7" containerName="istio-proxy" Oct 30 15:59:12.561518 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561501 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0fda42f-3851-4542-a220-b9591e30d2a7" containerName="istio-proxy" Oct 30 15:59:12.561518 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561509 5013 cpu_manager.go:414] "containerMap: removing container" podUID="6f1ad3ab-f617-4a25-a2c3-c47ba625624f" containerName="istio-proxy" Oct 30 15:59:12.561518 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561513 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f1ad3ab-f617-4a25-a2c3-c47ba625624f" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561524 5013 cpu_manager.go:414] "containerMap: removing container" podUID="13d1b70a-d4d0-472d-b0fd-df159b4b3be3" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561528 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13d1b70a-d4d0-472d-b0fd-df159b4b3be3" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561534 5013 cpu_manager.go:414] "containerMap: removing container" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561538 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561544 5013 cpu_manager.go:414] "containerMap: removing container" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561548 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561553 5013 cpu_manager.go:414] "containerMap: removing container" podUID="f0b22fc2-cafd-4d44-ab78-d1020db061fa" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561557 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0b22fc2-cafd-4d44-ab78-d1020db061fa" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561567 5013 cpu_manager.go:414] "containerMap: removing container" podUID="8821e53a-f78f-4139-ab3a-1cbafb6fe5a7" containerName="maas-api" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561571 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8821e53a-f78f-4139-ab3a-1cbafb6fe5a7" containerName="maas-api" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561577 5013 cpu_manager.go:414] "containerMap: removing container" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561581 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561587 5013 cpu_manager.go:414] "containerMap: removing container" podUID="46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561591 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561598 5013 cpu_manager.go:414] "containerMap: removing container" podUID="1b325488-f2d5-49c8-9c76-d70554d9f0fe" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561602 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b325488-f2d5-49c8-9c76-d70554d9f0fe" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561613 5013 cpu_manager.go:414] "containerMap: removing container" podUID="0cb01099-f910-4f21-a0b6-ff5dbc33bf60" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561618 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0cb01099-f910-4f21-a0b6-ff5dbc33bf60" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561625 5013 cpu_manager.go:414] "containerMap: removing container" podUID="04fbe630-342a-4ec9-91c6-48c89165b6de" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561629 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04fbe630-342a-4ec9-91c6-48c89165b6de" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561642 5013 cpu_manager.go:414] "containerMap: removing container" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561645 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561711 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="13d1b70a-d4d0-472d-b0fd-df159b4b3be3" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561718 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae0e7089-01f3-42b4-9590-51cdc0b4765a" containerName="main" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561723 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="017d73bc-9fcc-4c13-bfdd-af21ac301a65" containerName="main" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561729 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0b22fc2-cafd-4d44-ab78-d1020db061fa" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561735 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="8821e53a-f78f-4139-ab3a-1cbafb6fe5a7" containerName="maas-api" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561740 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9d63587-9839-45d5-8bbe-e5208b3446b6" containerName="main" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561744 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="46ca9dff-35b7-40c0-a2c1-ca1b0b77ff92" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561748 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b325488-f2d5-49c8-9c76-d70554d9f0fe" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561754 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="0cb01099-f910-4f21-a0b6-ff5dbc33bf60" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561760 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="04fbe630-342a-4ec9-91c6-48c89165b6de" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561766 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd53a00a-b214-47dd-8a64-af52d0e490a5" containerName="main" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561771 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="82fbbdd5-fc1a-421a-9bc2-865304cac190" containerName="main" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561776 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="83bf7cb0-3eae-41b2-8ba0-4db7b4634071" containerName="istio-proxy" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561780 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="af0c6ebb-6ae6-455e-9b35-e72ffb5538ba" containerName="main" Oct 30 15:59:12.561771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561787 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0fda42f-3851-4542-a220-b9591e30d2a7" containerName="istio-proxy" Oct 30 15:59:12.562832 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.561792 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f1ad3ab-f617-4a25-a2c3-c47ba625624f" containerName="istio-proxy" Oct 30 15:59:12.566254 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.566239 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.592018 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.591983 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z"] Oct 30 15:59:12.595188 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.595158 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.595335 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.595223 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djmws\" (UniqueName: \"kubernetes.io/projected/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-kube-api-access-djmws\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.595405 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.595331 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.595405 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.595356 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.595520 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.595447 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.595520 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.595481 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.595635 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.595610 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.595695 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.595652 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.595695 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.595682 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.696211 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.696173 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.696360 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.696232 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.696360 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.696293 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.696360 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.696310 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.696360 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.696351 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.696608 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.696382 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.696608 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.696412 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.696608 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.696496 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.696608 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.696528 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djmws\" (UniqueName: \"kubernetes.io/projected/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-kube-api-access-djmws\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.696608 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.696598 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.696866 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.696732 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.696866 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.696764 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.696940 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.696893 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.697037 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.697017 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.698831 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.698809 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.699022 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.699005 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.722527 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.722503 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djmws\" (UniqueName: \"kubernetes.io/projected/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-kube-api-access-djmws\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.722613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.722534 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:12.880649 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:12.880620 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:13.196572 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:13.196544 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z"] Oct 30 15:59:13.197950 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:59:13.197920 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ad1f5d_e58c_4949_839e_4f2fedc6f7c5.slice/crio-a22b76f21806243db84661a387895b44cbddb0979f0627e98a287ff6464e5970 WatchSource:0}: Error finding container a22b76f21806243db84661a387895b44cbddb0979f0627e98a287ff6464e5970: Status 404 returned error can't find the container with id a22b76f21806243db84661a387895b44cbddb0979f0627e98a287ff6464e5970 Oct 30 15:59:13.200287 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:13.200255 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:59:13.200357 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:13.200322 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:59:13.200357 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:13.200349 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:59:13.334049 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:13.334006 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" event={"ID":"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5","Type":"ContainerStarted","Data":"6a81096f9204bfe6a66727dd87a3f77c301b728338307cbba351decf5c4d2913"} Oct 30 15:59:13.334461 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:13.334077 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" event={"ID":"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5","Type":"ContainerStarted","Data":"a22b76f21806243db84661a387895b44cbddb0979f0627e98a287ff6464e5970"} Oct 30 15:59:13.381360 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:13.381295 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" podStartSLOduration=1.381278741 podStartE2EDuration="1.381278741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:59:13.37751978 +0000 UTC m=+1269.528416812" watchObservedRunningTime="2026-10-30 15:59:13.381278741 +0000 UTC m=+1269.532175773" Oct 30 15:59:13.884374 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:13.884325 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:14.100230 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:14.099637 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:14.884221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:14.883415 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 30 15:59:14.884221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:14.883492 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" podUID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 30 15:59:15.826218 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:15.825852 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6"] Oct 30 15:59:15.826656 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:15.826453 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" podUID="c49252c7-3414-4786-b79f-bba3a3593934" containerName="maas-api" containerID="cri-o://e016d46f9f2e98ed8515dd6b0dfa38ddd93de7ba8d6f8786c4291a17975ce38e" gracePeriod=30 Oct 30 15:59:15.914253 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:15.890603 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 30 15:59:15.914253 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:15.890661 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" podUID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 30 15:59:15.992573 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:15.992387 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:16.441478 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:16.441395 5013 generic.go:356] "Generic (PLEG): container finished" podID="c49252c7-3414-4786-b79f-bba3a3593934" containerID="e016d46f9f2e98ed8515dd6b0dfa38ddd93de7ba8d6f8786c4291a17975ce38e" exitCode=0 Oct 30 15:59:16.441706 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:16.441686 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" event={"ID":"c49252c7-3414-4786-b79f-bba3a3593934","Type":"ContainerDied","Data":"e016d46f9f2e98ed8515dd6b0dfa38ddd93de7ba8d6f8786c4291a17975ce38e"} Oct 30 15:59:16.463935 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:16.463900 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb"] Oct 30 15:59:16.464402 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:16.464372 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" podUID="4facca78-e175-41fe-8e57-5a70e801a751" containerName="maas-api" containerID="cri-o://1201f5e081673511312bc027288403837fd277724f9afba939759fd09faa5ac6" gracePeriod=30 Oct 30 15:59:16.589231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:16.575643 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:16.791269 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:16.765073 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:16.895631 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:16.895588 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 30 15:59:16.895879 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:16.895850 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" podUID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 30 15:59:16.965632 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:16.965596 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:17.791914 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:17.791878 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch"] Oct 30 15:59:17.812481 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:17.812436 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:17.875292 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:17.857512 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" podUID="2a71f35c-8390-40b1-b1d3-828d7330eed9" containerName="maas-api" containerID="cri-o://0e003aa8b03e1ed66a45db53d563a096c161a039f4fd8e3d229912a384f63087" gracePeriod=30 Oct 30 15:59:17.921223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:17.891371 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 30 15:59:17.921223 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:17.891453 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" podUID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 30 15:59:18.308222 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:18.291537 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:18.540231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:18.530306 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:18.540231 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:18.534659 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:59:18.685227 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:18.677554 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c49252c7-3414-4786-b79f-bba3a3593934-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c49252c7-3414-4786-b79f-bba3a3593934-maas-api-tls\") pod \"c49252c7-3414-4786-b79f-bba3a3593934\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " Oct 30 15:59:18.685227 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:18.677610 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c49252c7-3414-4786-b79f-bba3a3593934-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c49252c7-3414-4786-b79f-bba3a3593934-metrics-certs\") pod \"c49252c7-3414-4786-b79f-bba3a3593934\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " Oct 30 15:59:18.685227 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:18.677806 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c49252c7-3414-4786-b79f-bba3a3593934-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c49252c7-3414-4786-b79f-bba3a3593934-openshift-service-ca\") pod \"c49252c7-3414-4786-b79f-bba3a3593934\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " Oct 30 15:59:18.685227 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:18.677841 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c49252c7-3414-4786-b79f-bba3a3593934-kube-api-access-hhw2z\" (UniqueName: \"kubernetes.io/projected/c49252c7-3414-4786-b79f-bba3a3593934-kube-api-access-hhw2z\") pod \"c49252c7-3414-4786-b79f-bba3a3593934\" (UID: \"c49252c7-3414-4786-b79f-bba3a3593934\") " Oct 30 15:59:18.793803 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:18.792978 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c49252c7-3414-4786-b79f-bba3a3593934-openshift-service-ca" pod "c49252c7-3414-4786-b79f-bba3a3593934" (UID: "c49252c7-3414-4786-b79f-bba3a3593934"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:59:18.793803 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:18.793230 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:18.985217 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:18.956320 5013 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c49252c7-3414-4786-b79f-bba3a3593934-openshift-service-ca\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:18.985756 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:18.985711 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 30 15:59:18.985943 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:18.985915 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" podUID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 30 15:59:19.276219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.274819 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:19.306675 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.306485 5013 generic.go:356] "Generic (PLEG): container finished" podID="4facca78-e175-41fe-8e57-5a70e801a751" containerID="1201f5e081673511312bc027288403837fd277724f9afba939759fd09faa5ac6" exitCode=0 Oct 30 15:59:19.306675 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.306599 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" event={"ID":"4facca78-e175-41fe-8e57-5a70e801a751","Type":"ContainerDied","Data":"1201f5e081673511312bc027288403837fd277724f9afba939759fd09faa5ac6"} Oct 30 15:59:19.317884 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.317851 5013 generic.go:356] "Generic (PLEG): container finished" podID="2a71f35c-8390-40b1-b1d3-828d7330eed9" containerID="0e003aa8b03e1ed66a45db53d563a096c161a039f4fd8e3d229912a384f63087" exitCode=0 Oct 30 15:59:19.318545 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.318518 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" event={"ID":"2a71f35c-8390-40b1-b1d3-828d7330eed9","Type":"ContainerDied","Data":"0e003aa8b03e1ed66a45db53d563a096c161a039f4fd8e3d229912a384f63087"} Oct 30 15:59:19.479744 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.436348 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c49252c7-3414-4786-b79f-bba3a3593934-maas-api-tls" pod "c49252c7-3414-4786-b79f-bba3a3593934" (UID: "c49252c7-3414-4786-b79f-bba3a3593934"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:59:19.507349 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.507310 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c49252c7-3414-4786-b79f-bba3a3593934-metrics-certs" pod "c49252c7-3414-4786-b79f-bba3a3593934" (UID: "c49252c7-3414-4786-b79f-bba3a3593934"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:59:19.507629 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.507591 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c49252c7-3414-4786-b79f-bba3a3593934-kube-api-access-hhw2z" pod "c49252c7-3414-4786-b79f-bba3a3593934" (UID: "c49252c7-3414-4786-b79f-bba3a3593934"). InnerVolumeSpecName "kube-api-access-hhw2z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:19.553694 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.553655 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhw2z\" (UniqueName: \"kubernetes.io/projected/c49252c7-3414-4786-b79f-bba3a3593934-kube-api-access-hhw2z\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:19.553839 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.553697 5013 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c49252c7-3414-4786-b79f-bba3a3593934-maas-api-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:19.553839 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.553713 5013 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c49252c7-3414-4786-b79f-bba3a3593934-metrics-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:19.590591 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.589653 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" event={"ID":"c49252c7-3414-4786-b79f-bba3a3593934","Type":"ContainerDied","Data":"7ccafcbae29466dc9cd4d7be64702f7b141a2d2ddc7d7297d934decc590b89dc"} Oct 30 15:59:19.590591 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.589729 5013 scope.go:122] "RemoveContainer" containerID="e016d46f9f2e98ed8515dd6b0dfa38ddd93de7ba8d6f8786c4291a17975ce38e" Oct 30 15:59:19.590591 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.590013 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6" Oct 30 15:59:19.597684 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.597653 5013 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 15:59:19.690270 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.667582 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" podUID="f52a418d-1057-4580-8728-3fe24c4d308d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 15:59:19.690270 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.673739 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:59:19.768234 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.759148 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4facca78-e175-41fe-8e57-5a70e801a751-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4facca78-e175-41fe-8e57-5a70e801a751-openshift-service-ca\") pod \"4facca78-e175-41fe-8e57-5a70e801a751\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " Oct 30 15:59:19.768234 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.759330 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4facca78-e175-41fe-8e57-5a70e801a751-kube-api-access-cn6rv\" (UniqueName: \"kubernetes.io/projected/4facca78-e175-41fe-8e57-5a70e801a751-kube-api-access-cn6rv\") pod \"4facca78-e175-41fe-8e57-5a70e801a751\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " Oct 30 15:59:19.768234 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.759392 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4facca78-e175-41fe-8e57-5a70e801a751-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4facca78-e175-41fe-8e57-5a70e801a751-metrics-certs\") pod \"4facca78-e175-41fe-8e57-5a70e801a751\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " Oct 30 15:59:19.768234 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.759428 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4facca78-e175-41fe-8e57-5a70e801a751-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4facca78-e175-41fe-8e57-5a70e801a751-maas-api-tls\") pod \"4facca78-e175-41fe-8e57-5a70e801a751\" (UID: \"4facca78-e175-41fe-8e57-5a70e801a751\") " Oct 30 15:59:19.804401 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.804258 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:19.831377 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.831336 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4facca78-e175-41fe-8e57-5a70e801a751-openshift-service-ca" pod "4facca78-e175-41fe-8e57-5a70e801a751" (UID: "4facca78-e175-41fe-8e57-5a70e801a751"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:59:19.832301 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.832255 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4facca78-e175-41fe-8e57-5a70e801a751-maas-api-tls" pod "4facca78-e175-41fe-8e57-5a70e801a751" (UID: "4facca78-e175-41fe-8e57-5a70e801a751"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:59:19.834949 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.834888 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6"] Oct 30 15:59:19.835244 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.835189 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4facca78-e175-41fe-8e57-5a70e801a751-metrics-certs" pod "4facca78-e175-41fe-8e57-5a70e801a751" (UID: "4facca78-e175-41fe-8e57-5a70e801a751"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:59:19.837469 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.837441 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a2b3f1-798dfb86dd-rdfc6"] Oct 30 15:59:19.847919 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.847860 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4facca78-e175-41fe-8e57-5a70e801a751-kube-api-access-cn6rv" pod "4facca78-e175-41fe-8e57-5a70e801a751" (UID: "4facca78-e175-41fe-8e57-5a70e801a751"). InnerVolumeSpecName "kube-api-access-cn6rv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:19.848439 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.848261 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:59:19.861107 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.860999 5013 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4facca78-e175-41fe-8e57-5a70e801a751-openshift-service-ca\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:19.861107 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.861034 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn6rv\" (UniqueName: \"kubernetes.io/projected/4facca78-e175-41fe-8e57-5a70e801a751-kube-api-access-cn6rv\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:19.861107 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.861051 5013 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4facca78-e175-41fe-8e57-5a70e801a751-metrics-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:19.861107 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.861065 5013 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4facca78-e175-41fe-8e57-5a70e801a751-maas-api-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:19.888074 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.887649 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 30 15:59:19.888074 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.887704 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" podUID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 30 15:59:19.961729 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.961573 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2a71f35c-8390-40b1-b1d3-828d7330eed9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a71f35c-8390-40b1-b1d3-828d7330eed9-maas-api-tls\") pod \"2a71f35c-8390-40b1-b1d3-828d7330eed9\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " Oct 30 15:59:19.961729 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.961674 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a71f35c-8390-40b1-b1d3-828d7330eed9-kube-api-access-xtdc2\" (UniqueName: \"kubernetes.io/projected/2a71f35c-8390-40b1-b1d3-828d7330eed9-kube-api-access-xtdc2\") pod \"2a71f35c-8390-40b1-b1d3-828d7330eed9\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " Oct 30 15:59:20.009229 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.961788 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a71f35c-8390-40b1-b1d3-828d7330eed9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a71f35c-8390-40b1-b1d3-828d7330eed9-openshift-service-ca\") pod \"2a71f35c-8390-40b1-b1d3-828d7330eed9\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " Oct 30 15:59:20.009229 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.963012 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2a71f35c-8390-40b1-b1d3-828d7330eed9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a71f35c-8390-40b1-b1d3-828d7330eed9-metrics-certs\") pod \"2a71f35c-8390-40b1-b1d3-828d7330eed9\" (UID: \"2a71f35c-8390-40b1-b1d3-828d7330eed9\") " Oct 30 15:59:20.009229 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:19.964056 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a71f35c-8390-40b1-b1d3-828d7330eed9-openshift-service-ca" pod "2a71f35c-8390-40b1-b1d3-828d7330eed9" (UID: "2a71f35c-8390-40b1-b1d3-828d7330eed9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:59:20.049516 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.048255 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:20.065183 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.065148 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a71f35c-8390-40b1-b1d3-828d7330eed9-metrics-certs" pod "2a71f35c-8390-40b1-b1d3-828d7330eed9" (UID: "2a71f35c-8390-40b1-b1d3-828d7330eed9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:59:20.066551 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.066527 5013 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a71f35c-8390-40b1-b1d3-828d7330eed9-openshift-service-ca\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:20.066688 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.066674 5013 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a71f35c-8390-40b1-b1d3-828d7330eed9-metrics-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:20.099054 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.099017 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c49252c7-3414-4786-b79f-bba3a3593934" path="/var/lib/kubelet/pods/c49252c7-3414-4786-b79f-bba3a3593934/volumes" Oct 30 15:59:20.099296 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.099267 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a71f35c-8390-40b1-b1d3-828d7330eed9-maas-api-tls" pod "2a71f35c-8390-40b1-b1d3-828d7330eed9" (UID: "2a71f35c-8390-40b1-b1d3-828d7330eed9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:59:20.099873 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.099850 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a71f35c-8390-40b1-b1d3-828d7330eed9-kube-api-access-xtdc2" pod "2a71f35c-8390-40b1-b1d3-828d7330eed9" (UID: "2a71f35c-8390-40b1-b1d3-828d7330eed9"). InnerVolumeSpecName "kube-api-access-xtdc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:20.167765 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.167677 5013 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a71f35c-8390-40b1-b1d3-828d7330eed9-maas-api-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:20.167765 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.167711 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtdc2\" (UniqueName: \"kubernetes.io/projected/2a71f35c-8390-40b1-b1d3-828d7330eed9-kube-api-access-xtdc2\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:20.603547 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.603509 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" event={"ID":"4facca78-e175-41fe-8e57-5a70e801a751","Type":"ContainerDied","Data":"3ca3ca561c9b378799b22b04b8dce8b346811f178fb35b9ab185cfb9c379a3c8"} Oct 30 15:59:20.604004 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.603585 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb" Oct 30 15:59:20.604004 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.603593 5013 scope.go:122] "RemoveContainer" containerID="1201f5e081673511312bc027288403837fd277724f9afba939759fd09faa5ac6" Oct 30 15:59:20.605394 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.604970 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" Oct 30 15:59:20.605394 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.604994 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch" event={"ID":"2a71f35c-8390-40b1-b1d3-828d7330eed9","Type":"ContainerDied","Data":"878220ebd6844ab4e1f9fd5f979c3078bf840abd0786a40f694c4f2498dde33d"} Oct 30 15:59:20.613844 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.613829 5013 scope.go:122] "RemoveContainer" containerID="0e003aa8b03e1ed66a45db53d563a096c161a039f4fd8e3d229912a384f63087" Oct 30 15:59:20.655311 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.655289 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb"] Oct 30 15:59:20.667135 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.667096 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-040f9a-58f797c787-79jbb"] Oct 30 15:59:20.712771 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.712744 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch"] Oct 30 15:59:20.715282 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.715262 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-9e2b29-96d4b468f-rjsch"] Oct 30 15:59:20.883714 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.883252 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 30 15:59:20.883714 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:20.883315 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" podUID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 30 15:59:21.817357 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:21.817321 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:21.881601 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:21.881569 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 30 15:59:21.881773 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:21.881626 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" podUID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 30 15:59:22.007074 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:22.007039 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a71f35c-8390-40b1-b1d3-828d7330eed9" path="/var/lib/kubelet/pods/2a71f35c-8390-40b1-b1d3-828d7330eed9/volumes" Oct 30 15:59:22.007549 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:22.007534 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4facca78-e175-41fe-8e57-5a70e801a751" path="/var/lib/kubelet/pods/4facca78-e175-41fe-8e57-5a70e801a751/volumes" Oct 30 15:59:23.215551 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:23.215294 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:23.576717 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:23.576605 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:23.788219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:23.786868 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:23.901073 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:23.901042 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd"] Oct 30 15:59:23.901294 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:23.901261 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" podUID="04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" containerName="istio-proxy" containerID="cri-o://7e524e5979b2a11390fa39beb6ec1ed10a3e849055fa666d51e5fa5cbb4f3081" gracePeriod=30 Oct 30 15:59:23.925242 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:23.924097 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 15:59:23.925242 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:23.924184 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" podUID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 15:59:24.192449 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:24.192349 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:24.192867 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:24.192841 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:24.193736 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:24.193716 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 15:59:25.639030 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:25.638971 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:26.287507 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:26.287476 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:26.605248 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:26.603747 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:26.653261 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:26.653225 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv"] Oct 30 15:59:26.653787 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:26.653483 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" podUID="f52a418d-1057-4580-8728-3fe24c4d308d" containerName="istio-proxy" containerID="cri-o://d981588a7958e71322c571d89ad2070f1572720119846fcb3cc280fc8c21c59b" gracePeriod=30 Oct 30 15:59:26.663999 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:26.663978 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z"] Oct 30 15:59:26.664160 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:26.664142 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" podUID="ea1e0986-aa76-4df4-85ac-cf104e42fb7e" containerName="istio-proxy" containerID="cri-o://5acb53a976393e29bb46b7cb29feb5c459ab662969cb695d18258d6b43bcfe4f" gracePeriod=30 Oct 30 15:59:26.679107 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:26.679077 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46"] Oct 30 15:59:26.679332 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:26.679312 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" podUID="4e6a88f2-c53c-4346-81b6-68ea50a6528a" containerName="istio-proxy" containerID="cri-o://88a1e60d71eee58e70ec295441da04abb42fe59757e033ea0a4d8b5a646513fe" gracePeriod=30 Oct 30 15:59:26.796362 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:26.796328 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:26.978560 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:59:26.978522 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 15:59:27.157216 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.157162 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:27.382391 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.382364 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:27.473239 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.470614 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5"] Oct 30 15:59:27.473239 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.470933 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" podUID="b0f87920-2518-4aa2-9db4-5959b7e71a54" containerName="istio-proxy" containerID="cri-o://57f31ef6889321b22c87e27a59421508f1db555afdc72326ee9eb263ed1a8917" gracePeriod=30 Oct 30 15:59:27.570713 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.570663 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:27.619228 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.617172 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2"] Oct 30 15:59:27.619228 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.617958 5013 cpu_manager.go:414] "containerMap: removing container" podUID="c49252c7-3414-4786-b79f-bba3a3593934" containerName="maas-api" Oct 30 15:59:27.619228 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.617975 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c49252c7-3414-4786-b79f-bba3a3593934" containerName="maas-api" Oct 30 15:59:27.619228 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.618006 5013 cpu_manager.go:414] "containerMap: removing container" podUID="2a71f35c-8390-40b1-b1d3-828d7330eed9" containerName="maas-api" Oct 30 15:59:27.619228 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.618016 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a71f35c-8390-40b1-b1d3-828d7330eed9" containerName="maas-api" Oct 30 15:59:27.619228 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.618030 5013 cpu_manager.go:414] "containerMap: removing container" podUID="4facca78-e175-41fe-8e57-5a70e801a751" containerName="maas-api" Oct 30 15:59:27.619228 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.618040 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4facca78-e175-41fe-8e57-5a70e801a751" containerName="maas-api" Oct 30 15:59:27.627250 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.621393 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="c49252c7-3414-4786-b79f-bba3a3593934" containerName="maas-api" Oct 30 15:59:27.627250 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.621423 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a71f35c-8390-40b1-b1d3-828d7330eed9" containerName="maas-api" Oct 30 15:59:27.627250 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.621433 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="4facca78-e175-41fe-8e57-5a70e801a751" containerName="maas-api" Oct 30 15:59:27.629299 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.629047 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.651124 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.651096 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2"] Oct 30 15:59:27.760717 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.760680 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:27.777910 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.777875 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.778057 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.777937 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.778057 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.778011 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz6q6\" (UniqueName: \"kubernetes.io/projected/2ab3b174-5788-448b-b85f-49c879be5489-kube-api-access-zz6q6\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.778142 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.778070 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ab3b174-5788-448b-b85f-49c879be5489-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.778219 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.778173 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ab3b174-5788-448b-b85f-49c879be5489-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.778283 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.778235 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.778318 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.778297 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.778356 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.778331 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.778434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.778420 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ab3b174-5788-448b-b85f-49c879be5489-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.879908 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.879858 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ab3b174-5788-448b-b85f-49c879be5489-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.880088 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.879932 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.880088 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.879989 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.880088 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.880060 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.880270 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.880161 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ab3b174-5788-448b-b85f-49c879be5489-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.880270 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.880210 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.880434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.880267 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.880434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.880298 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz6q6\" (UniqueName: \"kubernetes.io/projected/2ab3b174-5788-448b-b85f-49c879be5489-kube-api-access-zz6q6\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.880434 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.880371 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ab3b174-5788-448b-b85f-49c879be5489-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.880591 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.880483 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.880591 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.880561 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.880712 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.880588 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.880775 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.880750 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.880960 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.880936 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ab3b174-5788-448b-b85f-49c879be5489-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.882447 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.882417 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.882866 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.882848 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ab3b174-5788-448b-b85f-49c879be5489-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.907040 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.906987 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ab3b174-5788-448b-b85f-49c879be5489-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.907327 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.907301 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz6q6\" (UniqueName: \"kubernetes.io/projected/2ab3b174-5788-448b-b85f-49c879be5489-kube-api-access-zz6q6\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.961241 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.961185 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:27.999080 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:27.999016 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:28.301870 ip-10-0-129-40 kubenswrapper[5013]: W1030 15:59:28.301843 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ab3b174_5788_448b_b85f_49c879be5489.slice/crio-c962a025c5b260f7beb884da672be867eccac700aef98623b02378e0fb2926e7 WatchSource:0}: Error finding container c962a025c5b260f7beb884da672be867eccac700aef98623b02378e0fb2926e7: Status 404 returned error can't find the container with id c962a025c5b260f7beb884da672be867eccac700aef98623b02378e0fb2926e7 Oct 30 15:59:28.302003 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:28.301956 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2"] Oct 30 15:59:28.304163 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:28.304128 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:59:28.304262 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:28.304213 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:59:28.304262 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:28.304246 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 15:59:28.665772 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:28.665742 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" event={"ID":"2ab3b174-5788-448b-b85f-49c879be5489","Type":"ContainerStarted","Data":"d0e2f475327b3a06ac5e995f35637b3f628c27c497d008b8364f52589b405118"} Oct 30 15:59:28.665927 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:28.665789 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" event={"ID":"2ab3b174-5788-448b-b85f-49c879be5489","Type":"ContainerStarted","Data":"c962a025c5b260f7beb884da672be867eccac700aef98623b02378e0fb2926e7"} Oct 30 15:59:28.907542 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:28.907492 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:28.962004 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:28.961972 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:29.141650 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.139767 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:29.238582 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.238555 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:59:29.292221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.292002 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" podStartSLOduration=2.291982215 podStartE2EDuration="2.291982215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:59:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:59:28.725386688 +0000 UTC m=+1284.876283719" watchObservedRunningTime="2026-10-30 15:59:29.291982215 +0000 UTC m=+1285.442879246" Oct 30 15:59:29.430178 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.430142 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-credential-socket\") pod \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " Oct 30 15:59:29.430362 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.430208 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-kube-api-access-v8rvb\" (UniqueName: \"kubernetes.io/projected/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-kube-api-access-v8rvb\") pod \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " Oct 30 15:59:29.430362 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.430267 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-workload-certs\") pod \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " Oct 30 15:59:29.430362 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.430287 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-podinfo\") pod \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " Oct 30 15:59:29.430362 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.430332 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-data\") pod \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " Oct 30 15:59:29.430556 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.430363 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-workload-socket\") pod \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " Oct 30 15:59:29.430556 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.430398 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-envoy\") pod \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " Oct 30 15:59:29.430556 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.430498 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istiod-ca-cert\") pod \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " Oct 30 15:59:29.430556 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.430511 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-credential-socket" pod "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" (UID: "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:29.430760 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.430573 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-token\" (UniqueName: \"kubernetes.io/projected/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-token\") pod \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\" (UID: \"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9\") " Oct 30 15:59:29.430760 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.430602 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-workload-certs" pod "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" (UID: "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:29.430760 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.430651 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-data" pod "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" (UID: "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:29.430923 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.430800 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-workload-socket" pod "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" (UID: "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:29.430983 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.430917 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istiod-ca-cert" pod "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" (UID: "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:59:29.431292 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.431266 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:29.431292 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.431292 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:29.431485 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.431307 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:29.431485 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.431321 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:29.431485 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.431333 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:29.432774 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.432751 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-kube-api-access-v8rvb" pod "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" (UID: "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9"). InnerVolumeSpecName "kube-api-access-v8rvb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:29.433047 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.433032 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-envoy" pod "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" (UID: "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:29.433258 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.433232 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-podinfo" pod "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" (UID: "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:59:29.433349 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.433303 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-token" pod "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" (UID: "04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:29.532745 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.532656 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8rvb\" (UniqueName: \"kubernetes.io/projected/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-kube-api-access-v8rvb\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:29.532745 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.532698 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:29.532745 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.532709 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:29.532745 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.532717 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:29.702259 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.700064 5013 generic.go:356] "Generic (PLEG): container finished" podID="04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" containerID="7e524e5979b2a11390fa39beb6ec1ed10a3e849055fa666d51e5fa5cbb4f3081" exitCode=0 Oct 30 15:59:29.702259 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.701014 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" Oct 30 15:59:29.702961 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.702531 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" event={"ID":"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9","Type":"ContainerDied","Data":"7e524e5979b2a11390fa39beb6ec1ed10a3e849055fa666d51e5fa5cbb4f3081"} Oct 30 15:59:29.702961 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.702598 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd" event={"ID":"04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9","Type":"ContainerDied","Data":"bf2be9765537f2527648dc85a72c788431086034892b7c823063a99d4c4346c7"} Oct 30 15:59:29.702961 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.702631 5013 scope.go:122] "RemoveContainer" containerID="7e524e5979b2a11390fa39beb6ec1ed10a3e849055fa666d51e5fa5cbb4f3081" Oct 30 15:59:29.715833 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.715814 5013 scope.go:122] "RemoveContainer" containerID="7e524e5979b2a11390fa39beb6ec1ed10a3e849055fa666d51e5fa5cbb4f3081" Oct 30 15:59:29.716323 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:59:29.716303 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e524e5979b2a11390fa39beb6ec1ed10a3e849055fa666d51e5fa5cbb4f3081\": container with ID starting with 7e524e5979b2a11390fa39beb6ec1ed10a3e849055fa666d51e5fa5cbb4f3081 not found: ID does not exist" containerID="7e524e5979b2a11390fa39beb6ec1ed10a3e849055fa666d51e5fa5cbb4f3081" Oct 30 15:59:29.716417 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.716330 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e524e5979b2a11390fa39beb6ec1ed10a3e849055fa666d51e5fa5cbb4f3081"} err="failed to get container status \"7e524e5979b2a11390fa39beb6ec1ed10a3e849055fa666d51e5fa5cbb4f3081\": rpc error: code = NotFound desc = could not find container \"7e524e5979b2a11390fa39beb6ec1ed10a3e849055fa666d51e5fa5cbb4f3081\": container with ID starting with 7e524e5979b2a11390fa39beb6ec1ed10a3e849055fa666d51e5fa5cbb4f3081 not found: ID does not exist" Oct 30 15:59:29.808391 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.808241 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd"] Oct 30 15:59:29.819483 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.819451 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-5s8sd"] Oct 30 15:59:29.906220 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.906082 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:29.963252 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.963223 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 15:59:29.963623 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:29.963287 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" podUID="2ab3b174-5788-448b-b85f-49c879be5489" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 15:59:30.009831 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:30.009793 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" path="/var/lib/kubelet/pods/04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9/volumes" Oct 30 15:59:30.159366 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:30.159336 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:30.962285 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:30.962248 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 15:59:30.962486 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:30.962315 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" podUID="2ab3b174-5788-448b-b85f-49c879be5489" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 15:59:31.468075 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:31.468036 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:31.719488 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:31.719333 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl"] Oct 30 15:59:31.719928 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:31.719894 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" podUID="ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" containerName="istio-proxy" containerID="cri-o://0955a0b0549ccf213519124395de6a34a9af8709494bf07c25403ebe67c48b00" gracePeriod=30 Oct 30 15:59:31.740247 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:31.739467 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5"] Oct 30 15:59:31.740247 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:31.739700 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" podUID="b477fa47-61b7-4232-a375-6ca81dfec7cf" containerName="istio-proxy" containerID="cri-o://6b87d2e48f4eff69d48e3c76a6e861b31a743f3d3503bdd93566e17cc81f6216" gracePeriod=30 Oct 30 15:59:31.743132 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:31.743066 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:31.758931 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:31.758896 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st"] Oct 30 15:59:31.759570 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:31.759408 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" podUID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" containerName="istio-proxy" containerID="cri-o://b8f302ae51f31a303cc92365bdbb551fb7dd8478a72ea6bf36d9b558237cf313" gracePeriod=30 Oct 30 15:59:31.962737 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:31.962697 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 15:59:31.962927 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:31.962905 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" podUID="2ab3b174-5788-448b-b85f-49c879be5489" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 15:59:32.011963 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.011865 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:32.055577 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.055551 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:59:32.064944 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.064922 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-credential-socket\") pod \"f52a418d-1057-4580-8728-3fe24c4d308d\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " Oct 30 15:59:32.065145 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.065128 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f52a418d-1057-4580-8728-3fe24c4d308d-istio-token\" (UniqueName: \"kubernetes.io/projected/f52a418d-1057-4580-8728-3fe24c4d308d-istio-token\") pod \"f52a418d-1057-4580-8728-3fe24c4d308d\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " Oct 30 15:59:32.065287 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.065275 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f52a418d-1057-4580-8728-3fe24c4d308d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52a418d-1057-4580-8728-3fe24c4d308d-istio-podinfo\") pod \"f52a418d-1057-4580-8728-3fe24c4d308d\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " Oct 30 15:59:32.065463 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.065187 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-credential-socket" pod "f52a418d-1057-4580-8728-3fe24c4d308d" (UID: "f52a418d-1057-4580-8728-3fe24c4d308d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.065583 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.065567 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-workload-certs\") pod \"f52a418d-1057-4580-8728-3fe24c4d308d\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " Oct 30 15:59:32.065670 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.065660 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f52a418d-1057-4580-8728-3fe24c4d308d-kube-api-access-djjnf\" (UniqueName: \"kubernetes.io/projected/f52a418d-1057-4580-8728-3fe24c4d308d-kube-api-access-djjnf\") pod \"f52a418d-1057-4580-8728-3fe24c4d308d\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " Oct 30 15:59:32.065756 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.065744 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f52a418d-1057-4580-8728-3fe24c4d308d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52a418d-1057-4580-8728-3fe24c4d308d-istiod-ca-cert\") pod \"f52a418d-1057-4580-8728-3fe24c4d308d\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " Oct 30 15:59:32.066094 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.066082 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-istio-envoy\") pod \"f52a418d-1057-4580-8728-3fe24c4d308d\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " Oct 30 15:59:32.066713 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.066276 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-workload-socket\") pod \"f52a418d-1057-4580-8728-3fe24c4d308d\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " Oct 30 15:59:32.066713 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.066308 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-istio-data\") pod \"f52a418d-1057-4580-8728-3fe24c4d308d\" (UID: \"f52a418d-1057-4580-8728-3fe24c4d308d\") " Oct 30 15:59:32.066713 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.065975 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-workload-certs" pod "f52a418d-1057-4580-8728-3fe24c4d308d" (UID: "f52a418d-1057-4580-8728-3fe24c4d308d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.066713 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.066026 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f52a418d-1057-4580-8728-3fe24c4d308d-istiod-ca-cert" pod "f52a418d-1057-4580-8728-3fe24c4d308d" (UID: "f52a418d-1057-4580-8728-3fe24c4d308d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:59:32.066713 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.066663 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-workload-socket" pod "f52a418d-1057-4580-8728-3fe24c4d308d" (UID: "f52a418d-1057-4580-8728-3fe24c4d308d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.067031 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.066749 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52a418d-1057-4580-8728-3fe24c4d308d-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.067031 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.066768 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.067031 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.066783 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.067031 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.066960 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-istio-data" pod "f52a418d-1057-4580-8728-3fe24c4d308d" (UID: "f52a418d-1057-4580-8728-3fe24c4d308d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.068837 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.068806 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f52a418d-1057-4580-8728-3fe24c4d308d-kube-api-access-djjnf" pod "f52a418d-1057-4580-8728-3fe24c4d308d" (UID: "f52a418d-1057-4580-8728-3fe24c4d308d"). InnerVolumeSpecName "kube-api-access-djjnf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:32.077930 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.077894 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f52a418d-1057-4580-8728-3fe24c4d308d-istio-token" pod "f52a418d-1057-4580-8728-3fe24c4d308d" (UID: "f52a418d-1057-4580-8728-3fe24c4d308d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:32.085967 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.085818 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-istio-envoy" pod "f52a418d-1057-4580-8728-3fe24c4d308d" (UID: "f52a418d-1057-4580-8728-3fe24c4d308d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.105415 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.088292 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f52a418d-1057-4580-8728-3fe24c4d308d-istio-podinfo" pod "f52a418d-1057-4580-8728-3fe24c4d308d" (UID: "f52a418d-1057-4580-8728-3fe24c4d308d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:59:32.124740 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.124717 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:59:32.135313 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.135291 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:59:32.167375 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167353 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-token\" (UniqueName: \"kubernetes.io/projected/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-token\") pod \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " Oct 30 15:59:32.167567 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167545 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-data\") pod \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " Oct 30 15:59:32.167690 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167594 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istiod-ca-cert\") pod \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " Oct 30 15:59:32.167690 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167628 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-workload-socket\") pod \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " Oct 30 15:59:32.167690 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167650 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e6a88f2-c53c-4346-81b6-68ea50a6528a-kube-api-access-n87fs\" (UniqueName: \"kubernetes.io/projected/4e6a88f2-c53c-4346-81b6-68ea50a6528a-kube-api-access-n87fs\") pod \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " Oct 30 15:59:32.167690 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167677 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istiod-ca-cert\") pod \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " Oct 30 15:59:32.167911 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167705 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-credential-socket\") pod \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " Oct 30 15:59:32.167911 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167751 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-podinfo\") pod \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " Oct 30 15:59:32.167911 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167780 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-workload-socket\") pod \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " Oct 30 15:59:32.167911 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167807 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-envoy\") pod \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " Oct 30 15:59:32.167911 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167845 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-credential-socket\") pod \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " Oct 30 15:59:32.167911 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167849 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-data" pod "4e6a88f2-c53c-4346-81b6-68ea50a6528a" (UID: "4e6a88f2-c53c-4346-81b6-68ea50a6528a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.167911 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167881 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-workload-certs\") pod \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " Oct 30 15:59:32.168155 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167915 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-envoy\") pod \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " Oct 30 15:59:32.168155 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167949 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-podinfo\") pod \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\" (UID: \"4e6a88f2-c53c-4346-81b6-68ea50a6528a\") " Oct 30 15:59:32.168155 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.167984 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-workload-certs\") pod \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " Oct 30 15:59:32.168155 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.168019 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-kube-api-access-qhm7f\" (UniqueName: \"kubernetes.io/projected/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-kube-api-access-qhm7f\") pod \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " Oct 30 15:59:32.168155 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.168069 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-token\" (UniqueName: \"kubernetes.io/projected/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-token\") pod \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " Oct 30 15:59:32.168155 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.168018 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istiod-ca-cert" pod "ea1e0986-aa76-4df4-85ac-cf104e42fb7e" (UID: "ea1e0986-aa76-4df4-85ac-cf104e42fb7e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:59:32.168155 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.168128 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-data\") pod \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\" (UID: \"ea1e0986-aa76-4df4-85ac-cf104e42fb7e\") " Oct 30 15:59:32.168155 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.168142 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-workload-socket" pod "ea1e0986-aa76-4df4-85ac-cf104e42fb7e" (UID: "ea1e0986-aa76-4df4-85ac-cf104e42fb7e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.168622 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.168318 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-workload-socket" pod "4e6a88f2-c53c-4346-81b6-68ea50a6528a" (UID: "4e6a88f2-c53c-4346-81b6-68ea50a6528a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.168622 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.168558 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-data" pod "ea1e0986-aa76-4df4-85ac-cf104e42fb7e" (UID: "ea1e0986-aa76-4df4-85ac-cf104e42fb7e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.168728 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.168685 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istiod-ca-cert" pod "4e6a88f2-c53c-4346-81b6-68ea50a6528a" (UID: "4e6a88f2-c53c-4346-81b6-68ea50a6528a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:59:32.168923 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.168898 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-credential-socket" pod "ea1e0986-aa76-4df4-85ac-cf104e42fb7e" (UID: "ea1e0986-aa76-4df4-85ac-cf104e42fb7e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.169034 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.168937 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-credential-socket" pod "4e6a88f2-c53c-4346-81b6-68ea50a6528a" (UID: "4e6a88f2-c53c-4346-81b6-68ea50a6528a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169220 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169247 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169264 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52a418d-1057-4580-8728-3fe24c4d308d-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169277 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52a418d-1057-4580-8728-3fe24c4d308d-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169292 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169305 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djjnf\" (UniqueName: \"kubernetes.io/projected/f52a418d-1057-4580-8728-3fe24c4d308d-kube-api-access-djjnf\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169323 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169336 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169376 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169389 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52a418d-1057-4580-8728-3fe24c4d308d-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169402 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169414 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169426 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169439 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.169613 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.169275 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-workload-certs" pod "4e6a88f2-c53c-4346-81b6-68ea50a6528a" (UID: "4e6a88f2-c53c-4346-81b6-68ea50a6528a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.192660 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.192307 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-workload-certs" pod "ea1e0986-aa76-4df4-85ac-cf104e42fb7e" (UID: "ea1e0986-aa76-4df4-85ac-cf104e42fb7e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.202241 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.200362 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-podinfo" pod "ea1e0986-aa76-4df4-85ac-cf104e42fb7e" (UID: "ea1e0986-aa76-4df4-85ac-cf104e42fb7e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:59:32.202241 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.200441 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-podinfo" pod "4e6a88f2-c53c-4346-81b6-68ea50a6528a" (UID: "4e6a88f2-c53c-4346-81b6-68ea50a6528a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:59:32.202241 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.200519 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-envoy" pod "ea1e0986-aa76-4df4-85ac-cf104e42fb7e" (UID: "ea1e0986-aa76-4df4-85ac-cf104e42fb7e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.202241 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.200633 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-token" pod "4e6a88f2-c53c-4346-81b6-68ea50a6528a" (UID: "4e6a88f2-c53c-4346-81b6-68ea50a6528a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:32.202241 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.201473 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-kube-api-access-qhm7f" pod "ea1e0986-aa76-4df4-85ac-cf104e42fb7e" (UID: "ea1e0986-aa76-4df4-85ac-cf104e42fb7e"). InnerVolumeSpecName "kube-api-access-qhm7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:32.202241 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.201580 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e6a88f2-c53c-4346-81b6-68ea50a6528a-kube-api-access-n87fs" pod "4e6a88f2-c53c-4346-81b6-68ea50a6528a" (UID: "4e6a88f2-c53c-4346-81b6-68ea50a6528a"). InnerVolumeSpecName "kube-api-access-n87fs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:32.203408 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.203380 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-token" pod "ea1e0986-aa76-4df4-85ac-cf104e42fb7e" (UID: "ea1e0986-aa76-4df4-85ac-cf104e42fb7e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:32.209020 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.208984 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-envoy" pod "4e6a88f2-c53c-4346-81b6-68ea50a6528a" (UID: "4e6a88f2-c53c-4346-81b6-68ea50a6528a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:32.296614 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.296398 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.296614 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.296429 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n87fs\" (UniqueName: \"kubernetes.io/projected/4e6a88f2-c53c-4346-81b6-68ea50a6528a-kube-api-access-n87fs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.296614 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.296444 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.296614 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.296466 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.296614 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.296479 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.296614 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.296490 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.296614 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.296503 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e6a88f2-c53c-4346-81b6-68ea50a6528a-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.296614 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.296516 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.296614 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.296529 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhm7f\" (UniqueName: \"kubernetes.io/projected/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-kube-api-access-qhm7f\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.296614 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.296541 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea1e0986-aa76-4df4-85ac-cf104e42fb7e-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:32.297924 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.297867 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:32.726999 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.726970 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:32.754061 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.754035 5013 generic.go:356] "Generic (PLEG): container finished" podID="b0f87920-2518-4aa2-9db4-5959b7e71a54" containerID="57f31ef6889321b22c87e27a59421508f1db555afdc72326ee9eb263ed1a8917" exitCode=0 Oct 30 15:59:32.754249 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.754115 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" event={"ID":"b0f87920-2518-4aa2-9db4-5959b7e71a54","Type":"ContainerDied","Data":"57f31ef6889321b22c87e27a59421508f1db555afdc72326ee9eb263ed1a8917"} Oct 30 15:59:32.755422 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.755401 5013 generic.go:356] "Generic (PLEG): container finished" podID="ea1e0986-aa76-4df4-85ac-cf104e42fb7e" containerID="5acb53a976393e29bb46b7cb29feb5c459ab662969cb695d18258d6b43bcfe4f" exitCode=0 Oct 30 15:59:32.755524 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.755448 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" Oct 30 15:59:32.755524 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.755479 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" event={"ID":"ea1e0986-aa76-4df4-85ac-cf104e42fb7e","Type":"ContainerDied","Data":"5acb53a976393e29bb46b7cb29feb5c459ab662969cb695d18258d6b43bcfe4f"} Oct 30 15:59:32.755597 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.755521 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z" event={"ID":"ea1e0986-aa76-4df4-85ac-cf104e42fb7e","Type":"ContainerDied","Data":"ff593709e3754c83b83d2ef4a31fea8ca8854d7db5667a257872715e469a3bec"} Oct 30 15:59:32.755597 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.755552 5013 scope.go:122] "RemoveContainer" containerID="5acb53a976393e29bb46b7cb29feb5c459ab662969cb695d18258d6b43bcfe4f" Oct 30 15:59:32.756845 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.756805 5013 generic.go:356] "Generic (PLEG): container finished" podID="4e6a88f2-c53c-4346-81b6-68ea50a6528a" containerID="88a1e60d71eee58e70ec295441da04abb42fe59757e033ea0a4d8b5a646513fe" exitCode=0 Oct 30 15:59:32.756918 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.756847 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" Oct 30 15:59:32.756918 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.756885 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" event={"ID":"4e6a88f2-c53c-4346-81b6-68ea50a6528a","Type":"ContainerDied","Data":"88a1e60d71eee58e70ec295441da04abb42fe59757e033ea0a4d8b5a646513fe"} Oct 30 15:59:32.757028 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.756943 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46" event={"ID":"4e6a88f2-c53c-4346-81b6-68ea50a6528a","Type":"ContainerDied","Data":"87bdc85aa4cf9d98954cd31d38debc8e7d14b1a917e63b7e4597e6a19edbade5"} Oct 30 15:59:32.758650 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.758615 5013 generic.go:356] "Generic (PLEG): container finished" podID="f52a418d-1057-4580-8728-3fe24c4d308d" containerID="d981588a7958e71322c571d89ad2070f1572720119846fcb3cc280fc8c21c59b" exitCode=0 Oct 30 15:59:32.758650 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.758644 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" event={"ID":"f52a418d-1057-4580-8728-3fe24c4d308d","Type":"ContainerDied","Data":"d981588a7958e71322c571d89ad2070f1572720119846fcb3cc280fc8c21c59b"} Oct 30 15:59:32.758856 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.758681 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" event={"ID":"f52a418d-1057-4580-8728-3fe24c4d308d","Type":"ContainerDied","Data":"dee1ea9dc80bf9cb97fbdd129031396ca99c19b7feaec84833bdfaedff56d5d2"} Oct 30 15:59:32.758856 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.758649 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv" Oct 30 15:59:32.766549 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.766526 5013 scope.go:122] "RemoveContainer" containerID="5acb53a976393e29bb46b7cb29feb5c459ab662969cb695d18258d6b43bcfe4f" Oct 30 15:59:32.766870 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:59:32.766838 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5acb53a976393e29bb46b7cb29feb5c459ab662969cb695d18258d6b43bcfe4f\": container with ID starting with 5acb53a976393e29bb46b7cb29feb5c459ab662969cb695d18258d6b43bcfe4f not found: ID does not exist" containerID="5acb53a976393e29bb46b7cb29feb5c459ab662969cb695d18258d6b43bcfe4f" Oct 30 15:59:32.766976 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.766867 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5acb53a976393e29bb46b7cb29feb5c459ab662969cb695d18258d6b43bcfe4f"} err="failed to get container status \"5acb53a976393e29bb46b7cb29feb5c459ab662969cb695d18258d6b43bcfe4f\": rpc error: code = NotFound desc = could not find container \"5acb53a976393e29bb46b7cb29feb5c459ab662969cb695d18258d6b43bcfe4f\": container with ID starting with 5acb53a976393e29bb46b7cb29feb5c459ab662969cb695d18258d6b43bcfe4f not found: ID does not exist" Oct 30 15:59:32.766976 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.766885 5013 scope.go:122] "RemoveContainer" containerID="88a1e60d71eee58e70ec295441da04abb42fe59757e033ea0a4d8b5a646513fe" Oct 30 15:59:32.803067 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.803041 5013 scope.go:122] "RemoveContainer" containerID="88a1e60d71eee58e70ec295441da04abb42fe59757e033ea0a4d8b5a646513fe" Oct 30 15:59:32.803403 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:59:32.803382 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88a1e60d71eee58e70ec295441da04abb42fe59757e033ea0a4d8b5a646513fe\": container with ID starting with 88a1e60d71eee58e70ec295441da04abb42fe59757e033ea0a4d8b5a646513fe not found: ID does not exist" containerID="88a1e60d71eee58e70ec295441da04abb42fe59757e033ea0a4d8b5a646513fe" Oct 30 15:59:32.803499 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.803411 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88a1e60d71eee58e70ec295441da04abb42fe59757e033ea0a4d8b5a646513fe"} err="failed to get container status \"88a1e60d71eee58e70ec295441da04abb42fe59757e033ea0a4d8b5a646513fe\": rpc error: code = NotFound desc = could not find container \"88a1e60d71eee58e70ec295441da04abb42fe59757e033ea0a4d8b5a646513fe\": container with ID starting with 88a1e60d71eee58e70ec295441da04abb42fe59757e033ea0a4d8b5a646513fe not found: ID does not exist" Oct 30 15:59:32.803499 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.803430 5013 scope.go:122] "RemoveContainer" containerID="d981588a7958e71322c571d89ad2070f1572720119846fcb3cc280fc8c21c59b" Oct 30 15:59:32.812379 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.812360 5013 scope.go:122] "RemoveContainer" containerID="d981588a7958e71322c571d89ad2070f1572720119846fcb3cc280fc8c21c59b" Oct 30 15:59:32.812673 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:59:32.812651 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d981588a7958e71322c571d89ad2070f1572720119846fcb3cc280fc8c21c59b\": container with ID starting with d981588a7958e71322c571d89ad2070f1572720119846fcb3cc280fc8c21c59b not found: ID does not exist" containerID="d981588a7958e71322c571d89ad2070f1572720119846fcb3cc280fc8c21c59b" Oct 30 15:59:32.812745 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.812684 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d981588a7958e71322c571d89ad2070f1572720119846fcb3cc280fc8c21c59b"} err="failed to get container status \"d981588a7958e71322c571d89ad2070f1572720119846fcb3cc280fc8c21c59b\": rpc error: code = NotFound desc = could not find container \"d981588a7958e71322c571d89ad2070f1572720119846fcb3cc280fc8c21c59b\": container with ID starting with d981588a7958e71322c571d89ad2070f1572720119846fcb3cc280fc8c21c59b not found: ID does not exist" Oct 30 15:59:32.828099 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.828075 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46"] Oct 30 15:59:32.840126 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.840106 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-nvh46"] Oct 30 15:59:32.899820 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.899763 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv"] Oct 30 15:59:32.902899 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.902876 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-c2fkv"] Oct 30 15:59:32.937964 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.937937 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z"] Oct 30 15:59:32.940643 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.940623 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-9lj8z"] Oct 30 15:59:32.962159 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.962132 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 15:59:32.962323 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.962235 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" podUID="2ab3b174-5788-448b-b85f-49c879be5489" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 15:59:32.964270 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:32.964185 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:33.203109 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.203086 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:59:33.307710 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.307629 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-podinfo\") pod \"b0f87920-2518-4aa2-9db4-5959b7e71a54\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " Oct 30 15:59:33.307710 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.307688 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-envoy\") pod \"b0f87920-2518-4aa2-9db4-5959b7e71a54\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " Oct 30 15:59:33.307953 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.307749 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-token\" (UniqueName: \"kubernetes.io/projected/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-token\") pod \"b0f87920-2518-4aa2-9db4-5959b7e71a54\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " Oct 30 15:59:33.307953 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.307804 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-credential-socket\") pod \"b0f87920-2518-4aa2-9db4-5959b7e71a54\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " Oct 30 15:59:33.307953 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.307866 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-workload-socket\") pod \"b0f87920-2518-4aa2-9db4-5959b7e71a54\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " Oct 30 15:59:33.307953 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.307923 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-data\") pod \"b0f87920-2518-4aa2-9db4-5959b7e71a54\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " Oct 30 15:59:33.308162 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.307956 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0f87920-2518-4aa2-9db4-5959b7e71a54-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0f87920-2518-4aa2-9db4-5959b7e71a54-istiod-ca-cert\") pod \"b0f87920-2518-4aa2-9db4-5959b7e71a54\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " Oct 30 15:59:33.308162 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.307990 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-workload-certs\") pod \"b0f87920-2518-4aa2-9db4-5959b7e71a54\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " Oct 30 15:59:33.308162 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.308053 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0f87920-2518-4aa2-9db4-5959b7e71a54-kube-api-access-jnkzr\" (UniqueName: \"kubernetes.io/projected/b0f87920-2518-4aa2-9db4-5959b7e71a54-kube-api-access-jnkzr\") pod \"b0f87920-2518-4aa2-9db4-5959b7e71a54\" (UID: \"b0f87920-2518-4aa2-9db4-5959b7e71a54\") " Oct 30 15:59:33.308386 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.308348 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-workload-socket" pod "b0f87920-2518-4aa2-9db4-5959b7e71a54" (UID: "b0f87920-2518-4aa2-9db4-5959b7e71a54"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:33.308518 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.308501 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-data" pod "b0f87920-2518-4aa2-9db4-5959b7e71a54" (UID: "b0f87920-2518-4aa2-9db4-5959b7e71a54"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:33.308633 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.308606 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-workload-certs" pod "b0f87920-2518-4aa2-9db4-5959b7e71a54" (UID: "b0f87920-2518-4aa2-9db4-5959b7e71a54"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:33.308633 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.308620 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-credential-socket" pod "b0f87920-2518-4aa2-9db4-5959b7e71a54" (UID: "b0f87920-2518-4aa2-9db4-5959b7e71a54"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:33.308762 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.308682 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0f87920-2518-4aa2-9db4-5959b7e71a54-istiod-ca-cert" pod "b0f87920-2518-4aa2-9db4-5959b7e71a54" (UID: "b0f87920-2518-4aa2-9db4-5959b7e71a54"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:59:33.308819 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.308801 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:33.308904 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.308820 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:33.308904 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.308835 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:33.310541 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.310516 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0f87920-2518-4aa2-9db4-5959b7e71a54-kube-api-access-jnkzr" pod "b0f87920-2518-4aa2-9db4-5959b7e71a54" (UID: "b0f87920-2518-4aa2-9db4-5959b7e71a54"). InnerVolumeSpecName "kube-api-access-jnkzr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:33.310689 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.310672 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-podinfo" pod "b0f87920-2518-4aa2-9db4-5959b7e71a54" (UID: "b0f87920-2518-4aa2-9db4-5959b7e71a54"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:59:33.311325 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.311303 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-token" pod "b0f87920-2518-4aa2-9db4-5959b7e71a54" (UID: "b0f87920-2518-4aa2-9db4-5959b7e71a54"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:33.315285 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.315264 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-envoy" pod "b0f87920-2518-4aa2-9db4-5959b7e71a54" (UID: "b0f87920-2518-4aa2-9db4-5959b7e71a54"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:33.410154 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.410122 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnkzr\" (UniqueName: \"kubernetes.io/projected/b0f87920-2518-4aa2-9db4-5959b7e71a54-kube-api-access-jnkzr\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:33.410154 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.410147 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:33.410154 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.410158 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:33.410154 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.410166 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0f87920-2518-4aa2-9db4-5959b7e71a54-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:33.410525 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.410176 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0f87920-2518-4aa2-9db4-5959b7e71a54-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:33.410525 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.410184 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0f87920-2518-4aa2-9db4-5959b7e71a54-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:33.767540 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.767510 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" Oct 30 15:59:33.767540 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.767534 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5" event={"ID":"b0f87920-2518-4aa2-9db4-5959b7e71a54","Type":"ContainerDied","Data":"9ac08a20fceae24c02f7e04fb5c92914b165b6de70fe1394d520e6c30fa3e5b0"} Oct 30 15:59:33.768052 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.767601 5013 scope.go:122] "RemoveContainer" containerID="57f31ef6889321b22c87e27a59421508f1db555afdc72326ee9eb263ed1a8917" Oct 30 15:59:33.844929 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.844894 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5"] Oct 30 15:59:33.851367 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.851341 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-rf4l5"] Oct 30 15:59:33.962476 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.962438 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 15:59:33.962642 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:33.962495 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" podUID="2ab3b174-5788-448b-b85f-49c879be5489" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 15:59:34.018108 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:34.018026 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e6a88f2-c53c-4346-81b6-68ea50a6528a" path="/var/lib/kubelet/pods/4e6a88f2-c53c-4346-81b6-68ea50a6528a/volumes" Oct 30 15:59:34.018465 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:34.018450 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0f87920-2518-4aa2-9db4-5959b7e71a54" path="/var/lib/kubelet/pods/b0f87920-2518-4aa2-9db4-5959b7e71a54/volumes" Oct 30 15:59:34.018883 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:34.018870 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea1e0986-aa76-4df4-85ac-cf104e42fb7e" path="/var/lib/kubelet/pods/ea1e0986-aa76-4df4-85ac-cf104e42fb7e/volumes" Oct 30 15:59:34.019619 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:34.019594 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f52a418d-1057-4580-8728-3fe24c4d308d" path="/var/lib/kubelet/pods/f52a418d-1057-4580-8728-3fe24c4d308d/volumes" Oct 30 15:59:34.114564 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:34.114523 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:34.151400 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:34.151366 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 15:59:34.151563 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:34.151435 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" podUID="ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 15:59:34.352376 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:34.352281 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:34.944858 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:34.944822 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:34.961890 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:34.961857 5013 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 15:59:34.962001 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:34.961916 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" podUID="2ab3b174-5788-448b-b85f-49c879be5489" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 15:59:35.965706 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:35.965673 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:35.966149 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:35.966070 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:35.966817 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:35.966797 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 15:59:36.199579 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:36.199544 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:36.817864 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:36.817828 5013 generic.go:356] "Generic (PLEG): container finished" podID="ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" containerID="0955a0b0549ccf213519124395de6a34a9af8709494bf07c25403ebe67c48b00" exitCode=0 Oct 30 15:59:36.818004 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:36.817878 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" event={"ID":"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa","Type":"ContainerDied","Data":"0955a0b0549ccf213519124395de6a34a9af8709494bf07c25403ebe67c48b00"} Oct 30 15:59:36.932865 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:36.932577 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:59:37.026843 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.026815 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:59:37.044110 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.044087 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:59:37.049181 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049164 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-kube-api-access-c9pq4\" (UniqueName: \"kubernetes.io/projected/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-kube-api-access-c9pq4\") pod \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " Oct 30 15:59:37.049286 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049225 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-data\") pod \"b477fa47-61b7-4232-a375-6ca81dfec7cf\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " Oct 30 15:59:37.049286 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049251 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-workload-socket\") pod \"b477fa47-61b7-4232-a375-6ca81dfec7cf\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " Oct 30 15:59:37.049286 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049268 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-podinfo\") pod \"b477fa47-61b7-4232-a375-6ca81dfec7cf\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " Oct 30 15:59:37.049453 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049332 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-workload-certs\") pod \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " Oct 30 15:59:37.049453 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049378 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b477fa47-61b7-4232-a375-6ca81dfec7cf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b477fa47-61b7-4232-a375-6ca81dfec7cf-istiod-ca-cert\") pod \"b477fa47-61b7-4232-a375-6ca81dfec7cf\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " Oct 30 15:59:37.049453 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049432 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-token\" (UniqueName: \"kubernetes.io/projected/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-token\") pod \"b477fa47-61b7-4232-a375-6ca81dfec7cf\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " Oct 30 15:59:37.049594 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049474 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-credential-socket\") pod \"b477fa47-61b7-4232-a375-6ca81dfec7cf\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " Oct 30 15:59:37.049594 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049529 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-envoy\") pod \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " Oct 30 15:59:37.049594 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049555 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-data\") pod \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " Oct 30 15:59:37.049594 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049562 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-workload-socket" pod "b477fa47-61b7-4232-a375-6ca81dfec7cf" (UID: "b477fa47-61b7-4232-a375-6ca81dfec7cf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.049594 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049583 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-envoy\") pod \"b477fa47-61b7-4232-a375-6ca81dfec7cf\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " Oct 30 15:59:37.049849 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049586 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-data" pod "b477fa47-61b7-4232-a375-6ca81dfec7cf" (UID: "b477fa47-61b7-4232-a375-6ca81dfec7cf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.049849 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049625 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-podinfo\") pod \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " Oct 30 15:59:37.049849 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049656 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b477fa47-61b7-4232-a375-6ca81dfec7cf-kube-api-access-v75sv\" (UniqueName: \"kubernetes.io/projected/b477fa47-61b7-4232-a375-6ca81dfec7cf-kube-api-access-v75sv\") pod \"b477fa47-61b7-4232-a375-6ca81dfec7cf\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " Oct 30 15:59:37.049849 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049685 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-workload-certs" pod "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" (UID: "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.049849 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049692 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-workload-socket\") pod \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " Oct 30 15:59:37.049849 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049809 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istiod-ca-cert\") pod \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " Oct 30 15:59:37.049849 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049830 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-credential-socket" pod "b477fa47-61b7-4232-a375-6ca81dfec7cf" (UID: "b477fa47-61b7-4232-a375-6ca81dfec7cf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.050274 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049867 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-workload-socket" pod "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" (UID: "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.050274 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.049887 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-workload-certs\") pod \"b477fa47-61b7-4232-a375-6ca81dfec7cf\" (UID: \"b477fa47-61b7-4232-a375-6ca81dfec7cf\") " Oct 30 15:59:37.050274 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.050039 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b477fa47-61b7-4232-a375-6ca81dfec7cf-istiod-ca-cert" pod "b477fa47-61b7-4232-a375-6ca81dfec7cf" (UID: "b477fa47-61b7-4232-a375-6ca81dfec7cf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:59:37.050274 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.050063 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-credential-socket\") pod \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " Oct 30 15:59:37.050274 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.050102 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-token\" (UniqueName: \"kubernetes.io/projected/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-token\") pod \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\" (UID: \"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa\") " Oct 30 15:59:37.050703 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.050563 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.050703 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.050583 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.050703 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.050598 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.050703 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.050611 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.050703 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.050625 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.050703 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.050638 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b477fa47-61b7-4232-a375-6ca81dfec7cf-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.052077 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.052052 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-envoy" pod "b477fa47-61b7-4232-a375-6ca81dfec7cf" (UID: "b477fa47-61b7-4232-a375-6ca81dfec7cf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.052168 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.052149 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-kube-api-access-c9pq4" pod "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" (UID: "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa"). InnerVolumeSpecName "kube-api-access-c9pq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:37.052705 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.052333 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-data" pod "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" (UID: "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.052705 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.052406 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-envoy" pod "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" (UID: "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.052705 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.052516 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-credential-socket" pod "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" (UID: "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.052705 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.052572 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-podinfo" pod "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" (UID: "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:59:37.052705 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.052582 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-token" pod "b477fa47-61b7-4232-a375-6ca81dfec7cf" (UID: "b477fa47-61b7-4232-a375-6ca81dfec7cf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:37.052705 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.052599 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istiod-ca-cert" pod "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" (UID: "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:59:37.052705 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.052698 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-workload-certs" pod "b477fa47-61b7-4232-a375-6ca81dfec7cf" (UID: "b477fa47-61b7-4232-a375-6ca81dfec7cf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.053027 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.053011 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-token" pod "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" (UID: "ef8ef827-bafc-4f76-96d3-4a5371e8bdaa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:37.053235 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.053177 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b477fa47-61b7-4232-a375-6ca81dfec7cf-kube-api-access-v75sv" pod "b477fa47-61b7-4232-a375-6ca81dfec7cf" (UID: "b477fa47-61b7-4232-a375-6ca81dfec7cf"). InnerVolumeSpecName "kube-api-access-v75sv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:37.053577 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.053563 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-podinfo" pod "b477fa47-61b7-4232-a375-6ca81dfec7cf" (UID: "b477fa47-61b7-4232-a375-6ca81dfec7cf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:59:37.151450 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.151418 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-data\") pod \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " Oct 30 15:59:37.151598 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.151479 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-workload-socket\") pod \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " Oct 30 15:59:37.151651 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.151618 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istiod-ca-cert\") pod \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " Oct 30 15:59:37.151702 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.151654 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a06cb93e-6948-474e-bc0c-6601b10e1e3d-kube-api-access-v5hkh\" (UniqueName: \"kubernetes.io/projected/a06cb93e-6948-474e-bc0c-6601b10e1e3d-kube-api-access-v5hkh\") pod \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " Oct 30 15:59:37.151702 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.151680 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-token\" (UniqueName: \"kubernetes.io/projected/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-token\") pod \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " Oct 30 15:59:37.151802 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.151708 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-envoy\") pod \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " Oct 30 15:59:37.151802 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.151705 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-workload-socket" pod "a06cb93e-6948-474e-bc0c-6601b10e1e3d" (UID: "a06cb93e-6948-474e-bc0c-6601b10e1e3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.151802 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.151713 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-data" pod "a06cb93e-6948-474e-bc0c-6601b10e1e3d" (UID: "a06cb93e-6948-474e-bc0c-6601b10e1e3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.151802 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.151751 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-workload-certs\") pod \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " Oct 30 15:59:37.152011 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.151802 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-credential-socket\") pod \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " Oct 30 15:59:37.152011 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.151842 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-podinfo\") pod \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\" (UID: \"a06cb93e-6948-474e-bc0c-6601b10e1e3d\") " Oct 30 15:59:37.152118 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152034 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-workload-certs" pod "a06cb93e-6948-474e-bc0c-6601b10e1e3d" (UID: "a06cb93e-6948-474e-bc0c-6601b10e1e3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.152118 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152078 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istiod-ca-cert" pod "a06cb93e-6948-474e-bc0c-6601b10e1e3d" (UID: "a06cb93e-6948-474e-bc0c-6601b10e1e3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 15:59:37.152365 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152308 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-credential-socket" pod "a06cb93e-6948-474e-bc0c-6601b10e1e3d" (UID: "a06cb93e-6948-474e-bc0c-6601b10e1e3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.152365 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152329 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152365 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152348 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152365 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152358 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152365 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152367 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152376 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152388 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152406 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9pq4\" (UniqueName: \"kubernetes.io/projected/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-kube-api-access-c9pq4\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152424 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152437 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152449 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152470 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152482 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152493 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152504 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b477fa47-61b7-4232-a375-6ca81dfec7cf-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152516 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.152878 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.152528 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v75sv\" (UniqueName: \"kubernetes.io/projected/b477fa47-61b7-4232-a375-6ca81dfec7cf-kube-api-access-v75sv\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.154589 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.154565 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-podinfo" pod "a06cb93e-6948-474e-bc0c-6601b10e1e3d" (UID: "a06cb93e-6948-474e-bc0c-6601b10e1e3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 15:59:37.155558 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.155524 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-envoy" pod "a06cb93e-6948-474e-bc0c-6601b10e1e3d" (UID: "a06cb93e-6948-474e-bc0c-6601b10e1e3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:37.156123 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.155562 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a06cb93e-6948-474e-bc0c-6601b10e1e3d-kube-api-access-v5hkh" pod "a06cb93e-6948-474e-bc0c-6601b10e1e3d" (UID: "a06cb93e-6948-474e-bc0c-6601b10e1e3d"). InnerVolumeSpecName "kube-api-access-v5hkh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:37.156950 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.156925 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-token" pod "a06cb93e-6948-474e-bc0c-6601b10e1e3d" (UID: "a06cb93e-6948-474e-bc0c-6601b10e1e3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:37.253797 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.253718 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.253797 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.253745 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.253797 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.253754 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5hkh\" (UniqueName: \"kubernetes.io/projected/a06cb93e-6948-474e-bc0c-6601b10e1e3d-kube-api-access-v5hkh\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.253797 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.253765 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.253797 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.253773 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a06cb93e-6948-474e-bc0c-6601b10e1e3d-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:37.826918 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.826885 5013 generic.go:356] "Generic (PLEG): container finished" podID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" containerID="b8f302ae51f31a303cc92365bdbb551fb7dd8478a72ea6bf36d9b558237cf313" exitCode=0 Oct 30 15:59:37.827100 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.826933 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" Oct 30 15:59:37.827100 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.826989 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" event={"ID":"a06cb93e-6948-474e-bc0c-6601b10e1e3d","Type":"ContainerDied","Data":"b8f302ae51f31a303cc92365bdbb551fb7dd8478a72ea6bf36d9b558237cf313"} Oct 30 15:59:37.827100 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.827052 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" event={"ID":"a06cb93e-6948-474e-bc0c-6601b10e1e3d","Type":"ContainerDied","Data":"8b92ece03a82978189e9328386f5c58d340e24923c4cfd110c7da1b91f633126"} Oct 30 15:59:37.827100 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.827084 5013 scope.go:122] "RemoveContainer" containerID="b8f302ae51f31a303cc92365bdbb551fb7dd8478a72ea6bf36d9b558237cf313" Oct 30 15:59:37.828696 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.828662 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" Oct 30 15:59:37.828881 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.828736 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl" event={"ID":"ef8ef827-bafc-4f76-96d3-4a5371e8bdaa","Type":"ContainerDied","Data":"750be0493de0219648d8282577692916debbe4e6f4dc1fdbd2ec000c553890a4"} Oct 30 15:59:37.830139 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.830115 5013 generic.go:356] "Generic (PLEG): container finished" podID="b477fa47-61b7-4232-a375-6ca81dfec7cf" containerID="6b87d2e48f4eff69d48e3c76a6e861b31a743f3d3503bdd93566e17cc81f6216" exitCode=0 Oct 30 15:59:37.830269 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.830172 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" Oct 30 15:59:37.830339 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.830311 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" event={"ID":"b477fa47-61b7-4232-a375-6ca81dfec7cf","Type":"ContainerDied","Data":"6b87d2e48f4eff69d48e3c76a6e861b31a743f3d3503bdd93566e17cc81f6216"} Oct 30 15:59:37.830395 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.830353 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5" event={"ID":"b477fa47-61b7-4232-a375-6ca81dfec7cf","Type":"ContainerDied","Data":"603ca00e72c07bd162330e7375f8328a397c5d7d6908703fa263159898d2d2bd"} Oct 30 15:59:37.838117 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.837884 5013 scope.go:122] "RemoveContainer" containerID="b8f302ae51f31a303cc92365bdbb551fb7dd8478a72ea6bf36d9b558237cf313" Oct 30 15:59:37.838228 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:59:37.838178 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8f302ae51f31a303cc92365bdbb551fb7dd8478a72ea6bf36d9b558237cf313\": container with ID starting with b8f302ae51f31a303cc92365bdbb551fb7dd8478a72ea6bf36d9b558237cf313 not found: ID does not exist" containerID="b8f302ae51f31a303cc92365bdbb551fb7dd8478a72ea6bf36d9b558237cf313" Oct 30 15:59:37.838401 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.838235 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8f302ae51f31a303cc92365bdbb551fb7dd8478a72ea6bf36d9b558237cf313"} err="failed to get container status \"b8f302ae51f31a303cc92365bdbb551fb7dd8478a72ea6bf36d9b558237cf313\": rpc error: code = NotFound desc = could not find container \"b8f302ae51f31a303cc92365bdbb551fb7dd8478a72ea6bf36d9b558237cf313\": container with ID starting with b8f302ae51f31a303cc92365bdbb551fb7dd8478a72ea6bf36d9b558237cf313 not found: ID does not exist" Oct 30 15:59:37.838401 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.838257 5013 scope.go:122] "RemoveContainer" containerID="0955a0b0549ccf213519124395de6a34a9af8709494bf07c25403ebe67c48b00" Oct 30 15:59:37.848156 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.848140 5013 scope.go:122] "RemoveContainer" containerID="6b87d2e48f4eff69d48e3c76a6e861b31a743f3d3503bdd93566e17cc81f6216" Oct 30 15:59:37.856902 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.856838 5013 scope.go:122] "RemoveContainer" containerID="6b87d2e48f4eff69d48e3c76a6e861b31a743f3d3503bdd93566e17cc81f6216" Oct 30 15:59:37.857170 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:59:37.857146 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b87d2e48f4eff69d48e3c76a6e861b31a743f3d3503bdd93566e17cc81f6216\": container with ID starting with 6b87d2e48f4eff69d48e3c76a6e861b31a743f3d3503bdd93566e17cc81f6216 not found: ID does not exist" containerID="6b87d2e48f4eff69d48e3c76a6e861b31a743f3d3503bdd93566e17cc81f6216" Oct 30 15:59:37.857271 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.857181 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b87d2e48f4eff69d48e3c76a6e861b31a743f3d3503bdd93566e17cc81f6216"} err="failed to get container status \"6b87d2e48f4eff69d48e3c76a6e861b31a743f3d3503bdd93566e17cc81f6216\": rpc error: code = NotFound desc = could not find container \"6b87d2e48f4eff69d48e3c76a6e861b31a743f3d3503bdd93566e17cc81f6216\": container with ID starting with 6b87d2e48f4eff69d48e3c76a6e861b31a743f3d3503bdd93566e17cc81f6216 not found: ID does not exist" Oct 30 15:59:37.895145 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.895117 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st"] Oct 30 15:59:37.907267 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.907238 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st"] Oct 30 15:59:37.936354 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.936330 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl"] Oct 30 15:59:37.938235 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.938215 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-flbwl"] Oct 30 15:59:37.971121 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.971100 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5"] Oct 30 15:59:37.975389 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:37.975365 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-mknv5"] Oct 30 15:59:38.006241 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:38.006173 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" path="/var/lib/kubelet/pods/a06cb93e-6948-474e-bc0c-6601b10e1e3d/volumes" Oct 30 15:59:38.006634 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:38.006619 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b477fa47-61b7-4232-a375-6ca81dfec7cf" path="/var/lib/kubelet/pods/b477fa47-61b7-4232-a375-6ca81dfec7cf/volumes" Oct 30 15:59:38.007097 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:38.007082 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" path="/var/lib/kubelet/pods/ef8ef827-bafc-4f76-96d3-4a5371e8bdaa/volumes" Oct 30 15:59:38.035879 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:38.035855 5013 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 15:59:38.036221 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:38.035902 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-7k5st" podUID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 15:59:38.205520 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:38.205473 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:39.856693 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:39.856661 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:40.018475 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:40.018445 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c"] Oct 30 15:59:40.171944 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:40.171887 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:40.352416 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:40.352385 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:41.914920 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:41.914892 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:43.800248 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:43.800121 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:45.624508 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:45.624478 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:45.797713 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:45.797682 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:45.976990 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:45.976927 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:55.038958 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.038900 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" containerID="cri-o://6d1fdff51d12f6704bb19a41d8b2a94e7f926bcde3b7fca9cbce469153650de3" gracePeriod=45 Oct 30 15:59:55.154350 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.154313 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": dial tcp 10.132.0.53:8000: connect: connection refused" Oct 30 15:59:55.310033 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.310010 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:59:55.446447 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.446419 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-tmp-dir\") pod \"c636d9c4-4262-46ba-9fd4-64388f70a905\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " Oct 30 15:59:55.446627 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.446465 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-model-cache\") pod \"c636d9c4-4262-46ba-9fd4-64388f70a905\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " Oct 30 15:59:55.446627 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.446486 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-home\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-home\") pod \"c636d9c4-4262-46ba-9fd4-64388f70a905\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " Oct 30 15:59:55.446627 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.446563 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-dshm\") pod \"c636d9c4-4262-46ba-9fd4-64388f70a905\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " Oct 30 15:59:55.446627 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.446593 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c636d9c4-4262-46ba-9fd4-64388f70a905-kube-api-access-csshf\" (UniqueName: \"kubernetes.io/projected/c636d9c4-4262-46ba-9fd4-64388f70a905-kube-api-access-csshf\") pod \"c636d9c4-4262-46ba-9fd4-64388f70a905\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " Oct 30 15:59:55.446834 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.446637 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c636d9c4-4262-46ba-9fd4-64388f70a905-tls-certs\" (UniqueName: \"kubernetes.io/secret/c636d9c4-4262-46ba-9fd4-64388f70a905-tls-certs\") pod \"c636d9c4-4262-46ba-9fd4-64388f70a905\" (UID: \"c636d9c4-4262-46ba-9fd4-64388f70a905\") " Oct 30 15:59:55.446834 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.446727 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-model-cache" pod "c636d9c4-4262-46ba-9fd4-64388f70a905" (UID: "c636d9c4-4262-46ba-9fd4-64388f70a905"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:55.446834 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.446764 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-tmp-dir" pod "c636d9c4-4262-46ba-9fd4-64388f70a905" (UID: "c636d9c4-4262-46ba-9fd4-64388f70a905"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:55.446834 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.446788 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-home" pod "c636d9c4-4262-46ba-9fd4-64388f70a905" (UID: "c636d9c4-4262-46ba-9fd4-64388f70a905"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:55.447013 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.446956 5013 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-tmp-dir\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:55.447013 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.446974 5013 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-model-cache\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:55.447013 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.446988 5013 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-home\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:55.448976 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.448946 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c636d9c4-4262-46ba-9fd4-64388f70a905-tls-certs" pod "c636d9c4-4262-46ba-9fd4-64388f70a905" (UID: "c636d9c4-4262-46ba-9fd4-64388f70a905"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 15:59:55.448976 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.448970 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-dshm" pod "c636d9c4-4262-46ba-9fd4-64388f70a905" (UID: "c636d9c4-4262-46ba-9fd4-64388f70a905"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 15:59:55.449125 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.449022 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c636d9c4-4262-46ba-9fd4-64388f70a905-kube-api-access-csshf" pod "c636d9c4-4262-46ba-9fd4-64388f70a905" (UID: "c636d9c4-4262-46ba-9fd4-64388f70a905"). InnerVolumeSpecName "kube-api-access-csshf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 15:59:55.548469 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.548401 5013 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c636d9c4-4262-46ba-9fd4-64388f70a905-dshm\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:55.548469 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.548426 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csshf\" (UniqueName: \"kubernetes.io/projected/c636d9c4-4262-46ba-9fd4-64388f70a905-kube-api-access-csshf\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:55.548469 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.548435 5013 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c636d9c4-4262-46ba-9fd4-64388f70a905-tls-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 15:59:55.903580 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.903549 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 15:59:55.930692 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.930667 5013 generic.go:356] "Generic (PLEG): container finished" podID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerID="6d1fdff51d12f6704bb19a41d8b2a94e7f926bcde3b7fca9cbce469153650de3" exitCode=0 Oct 30 15:59:55.930832 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.930700 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" event={"ID":"c636d9c4-4262-46ba-9fd4-64388f70a905","Type":"ContainerDied","Data":"6d1fdff51d12f6704bb19a41d8b2a94e7f926bcde3b7fca9cbce469153650de3"} Oct 30 15:59:55.930832 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.930710 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" Oct 30 15:59:55.930832 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.930753 5013 scope.go:122] "RemoveContainer" containerID="6d1fdff51d12f6704bb19a41d8b2a94e7f926bcde3b7fca9cbce469153650de3" Oct 30 15:59:55.930989 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.930743 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c" event={"ID":"c636d9c4-4262-46ba-9fd4-64388f70a905","Type":"ContainerDied","Data":"28478d13f0a4cd675bfd926d07df35e11cd7dcfcec9ec6017b174a351cf06d3a"} Oct 30 15:59:55.940271 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.940257 5013 scope.go:122] "RemoveContainer" containerID="6d1fdff51d12f6704bb19a41d8b2a94e7f926bcde3b7fca9cbce469153650de3" Oct 30 15:59:55.940536 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:59:55.940518 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d1fdff51d12f6704bb19a41d8b2a94e7f926bcde3b7fca9cbce469153650de3\": container with ID starting with 6d1fdff51d12f6704bb19a41d8b2a94e7f926bcde3b7fca9cbce469153650de3 not found: ID does not exist" containerID="6d1fdff51d12f6704bb19a41d8b2a94e7f926bcde3b7fca9cbce469153650de3" Oct 30 15:59:55.940590 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.940544 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d1fdff51d12f6704bb19a41d8b2a94e7f926bcde3b7fca9cbce469153650de3"} err="failed to get container status \"6d1fdff51d12f6704bb19a41d8b2a94e7f926bcde3b7fca9cbce469153650de3\": rpc error: code = NotFound desc = could not find container \"6d1fdff51d12f6704bb19a41d8b2a94e7f926bcde3b7fca9cbce469153650de3\": container with ID starting with 6d1fdff51d12f6704bb19a41d8b2a94e7f926bcde3b7fca9cbce469153650de3 not found: ID does not exist" Oct 30 15:59:55.979600 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.979576 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c"] Oct 30 15:59:55.990882 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:55.990861 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/simulator-w5-ba9db9-kserve-7dcd64c7f-2w82c"] Oct 30 15:59:56.004552 ip-10-0-129-40 kubenswrapper[5013]: I1030 15:59:56.004521 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" path="/var/lib/kubelet/pods/c636d9c4-4262-46ba-9fd4-64388f70a905/volumes" Oct 30 15:59:56.975631 ip-10-0-129-40 kubenswrapper[5013]: E1030 15:59:56.975594 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:00:02.206569 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.206528 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j"] Oct 30 16:00:02.215920 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.215878 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" podUID="1f037ba0-aee4-4f9e-8b48-a3e7c1520671" containerName="maas-api" containerID="cri-o://242c362c573a6076f8b7b85902f6caa1b9db909a7c6f142c3d26e6246b8ce4ec" gracePeriod=30 Oct 30 16:00:02.370810 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.370730 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:02.573137 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.573111 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 16:00:02.671367 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.671335 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-maas-api-tls\") pod \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " Oct 30 16:00:02.671541 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.671394 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-metrics-certs\") pod \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " Oct 30 16:00:02.671541 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.671474 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-kube-api-access-jrcmk\" (UniqueName: \"kubernetes.io/projected/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-kube-api-access-jrcmk\") pod \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " Oct 30 16:00:02.671541 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.671524 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-openshift-service-ca\") pod \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\" (UID: \"1f037ba0-aee4-4f9e-8b48-a3e7c1520671\") " Oct 30 16:00:02.671988 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.671952 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-openshift-service-ca" pod "1f037ba0-aee4-4f9e-8b48-a3e7c1520671" (UID: "1f037ba0-aee4-4f9e-8b48-a3e7c1520671"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 16:00:02.673653 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.673622 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-maas-api-tls" pod "1f037ba0-aee4-4f9e-8b48-a3e7c1520671" (UID: "1f037ba0-aee4-4f9e-8b48-a3e7c1520671"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 16:00:02.673749 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.673688 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-kube-api-access-jrcmk" pod "1f037ba0-aee4-4f9e-8b48-a3e7c1520671" (UID: "1f037ba0-aee4-4f9e-8b48-a3e7c1520671"). InnerVolumeSpecName "kube-api-access-jrcmk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 16:00:02.673787 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.673745 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-metrics-certs" pod "1f037ba0-aee4-4f9e-8b48-a3e7c1520671" (UID: "1f037ba0-aee4-4f9e-8b48-a3e7c1520671"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 16:00:02.772968 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.772881 5013 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-metrics-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:02.772968 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.772920 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrcmk\" (UniqueName: \"kubernetes.io/projected/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-kube-api-access-jrcmk\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:02.772968 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.772935 5013 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-openshift-service-ca\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:02.772968 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.772948 5013 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f037ba0-aee4-4f9e-8b48-a3e7c1520671-maas-api-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:02.935221 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.935162 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:02.967604 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.967574 5013 generic.go:356] "Generic (PLEG): container finished" podID="1f037ba0-aee4-4f9e-8b48-a3e7c1520671" containerID="242c362c573a6076f8b7b85902f6caa1b9db909a7c6f142c3d26e6246b8ce4ec" exitCode=0 Oct 30 16:00:02.967751 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.967625 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" Oct 30 16:00:02.967751 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.967635 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" event={"ID":"1f037ba0-aee4-4f9e-8b48-a3e7c1520671","Type":"ContainerDied","Data":"242c362c573a6076f8b7b85902f6caa1b9db909a7c6f142c3d26e6246b8ce4ec"} Oct 30 16:00:02.967751 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.967684 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j" event={"ID":"1f037ba0-aee4-4f9e-8b48-a3e7c1520671","Type":"ContainerDied","Data":"b5b1d2a49ef7b6f9925f270d886bf8f66ca51f221577715491d516954c36700b"} Oct 30 16:00:02.967751 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.967705 5013 scope.go:122] "RemoveContainer" containerID="242c362c573a6076f8b7b85902f6caa1b9db909a7c6f142c3d26e6246b8ce4ec" Oct 30 16:00:02.977735 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.977720 5013 scope.go:122] "RemoveContainer" containerID="242c362c573a6076f8b7b85902f6caa1b9db909a7c6f142c3d26e6246b8ce4ec" Oct 30 16:00:02.978003 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:00:02.977979 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"242c362c573a6076f8b7b85902f6caa1b9db909a7c6f142c3d26e6246b8ce4ec\": container with ID starting with 242c362c573a6076f8b7b85902f6caa1b9db909a7c6f142c3d26e6246b8ce4ec not found: ID does not exist" containerID="242c362c573a6076f8b7b85902f6caa1b9db909a7c6f142c3d26e6246b8ce4ec" Oct 30 16:00:02.978057 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:02.978004 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"242c362c573a6076f8b7b85902f6caa1b9db909a7c6f142c3d26e6246b8ce4ec"} err="failed to get container status \"242c362c573a6076f8b7b85902f6caa1b9db909a7c6f142c3d26e6246b8ce4ec\": rpc error: code = NotFound desc = could not find container \"242c362c573a6076f8b7b85902f6caa1b9db909a7c6f142c3d26e6246b8ce4ec\": container with ID starting with 242c362c573a6076f8b7b85902f6caa1b9db909a7c6f142c3d26e6246b8ce4ec not found: ID does not exist" Oct 30 16:00:03.034725 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:03.034644 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j"] Oct 30 16:00:03.041452 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:03.041426 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-ba9db9-68dc855c56-rm72j"] Oct 30 16:00:03.121265 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:03.121235 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:03.749927 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:03.749896 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:04.005891 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:04.005820 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f037ba0-aee4-4f9e-8b48-a3e7c1520671" path="/var/lib/kubelet/pods/1f037ba0-aee4-4f9e-8b48-a3e7c1520671/volumes" Oct 30 16:00:09.879949 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:09.879902 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:10.055140 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:10.055106 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:10.160148 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:10.160070 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z"] Oct 30 16:00:10.160386 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:10.160363 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" podUID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" containerName="istio-proxy" containerID="cri-o://6a81096f9204bfe6a66727dd87a3f77c301b728338307cbba351decf5c4d2913" gracePeriod=30 Oct 30 16:00:10.172571 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:10.172546 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2"] Oct 30 16:00:10.172748 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:10.172722 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" podUID="2ab3b174-5788-448b-b85f-49c879be5489" containerName="istio-proxy" containerID="cri-o://d0e2f475327b3a06ac5e995f35637b3f628c27c497d008b8364f52589b405118" gracePeriod=30 Oct 30 16:00:10.181398 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:10.181371 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t"] Oct 30 16:00:10.181602 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:10.181580 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" podUID="91ed8818-5236-42e2-92f5-74e29232e3e5" containerName="istio-proxy" containerID="cri-o://8761b289dee89aa1f9575cbf1c5e82dc33d096463fe3d8abd63359749176ab90" gracePeriod=30 Oct 30 16:00:10.336788 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:10.336745 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:15.035285 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.035251 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5964675dcd-lswsg"] Oct 30 16:00:15.035769 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.035476 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-5964675dcd-lswsg" podUID="5adc0947-063e-4c71-a2c6-d8395cc3b0d2" containerName="manager" containerID="cri-o://19110742045fe84ab2b11e12484af32385528c38e0148b5dcc58fe02a168da7f" gracePeriod=30 Oct 30 16:00:15.351527 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.351497 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 16:00:15.381617 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.381589 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-webhook-cert\" (UniqueName: \"kubernetes.io/secret/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-webhook-cert\") pod \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\" (UID: \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\") " Oct 30 16:00:15.381785 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.381632 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-metrics-certs\") pod \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\" (UID: \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\") " Oct 30 16:00:15.381785 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.381768 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-kube-api-access-44zf9\" (UniqueName: \"kubernetes.io/projected/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-kube-api-access-44zf9\") pod \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\" (UID: \"5adc0947-063e-4c71-a2c6-d8395cc3b0d2\") " Oct 30 16:00:15.384716 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.384682 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-kube-api-access-44zf9" pod "5adc0947-063e-4c71-a2c6-d8395cc3b0d2" (UID: "5adc0947-063e-4c71-a2c6-d8395cc3b0d2"). InnerVolumeSpecName "kube-api-access-44zf9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 16:00:15.384868 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.384720 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-metrics-certs" pod "5adc0947-063e-4c71-a2c6-d8395cc3b0d2" (UID: "5adc0947-063e-4c71-a2c6-d8395cc3b0d2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 16:00:15.385015 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.384991 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-webhook-cert" pod "5adc0947-063e-4c71-a2c6-d8395cc3b0d2" (UID: "5adc0947-063e-4c71-a2c6-d8395cc3b0d2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 16:00:15.432950 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.432858 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 16:00:15.483794 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.482646 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-credential-socket\") pod \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " Oct 30 16:00:15.483794 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.482724 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-workload-certs\") pod \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " Oct 30 16:00:15.483794 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.482773 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istiod-ca-cert\") pod \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " Oct 30 16:00:15.483794 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.482803 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-workload-socket\") pod \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " Oct 30 16:00:15.483794 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.482869 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-token\" (UniqueName: \"kubernetes.io/projected/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-token\") pod \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " Oct 30 16:00:15.483794 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.482903 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-envoy\") pod \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " Oct 30 16:00:15.483794 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.482990 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-data\") pod \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " Oct 30 16:00:15.483794 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.483022 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-podinfo\") pod \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " Oct 30 16:00:15.483794 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.483075 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-kube-api-access-djmws\" (UniqueName: \"kubernetes.io/projected/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-kube-api-access-djmws\") pod \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\" (UID: \"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5\") " Oct 30 16:00:15.483794 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.483380 5013 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-webhook-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.483794 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.483396 5013 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-metrics-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.483794 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.483409 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44zf9\" (UniqueName: \"kubernetes.io/projected/5adc0947-063e-4c71-a2c6-d8395cc3b0d2-kube-api-access-44zf9\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.484698 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.484490 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-workload-certs" pod "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" (UID: "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.489069 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.489041 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-workload-socket" pod "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" (UID: "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.489371 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.489352 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istiod-ca-cert" pod "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" (UID: "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 16:00:15.490410 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.490381 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-data" pod "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" (UID: "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.490525 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.490471 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-credential-socket" pod "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" (UID: "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.492210 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.492159 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-envoy" pod "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" (UID: "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.492943 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.492906 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-token" pod "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" (UID: "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 16:00:15.494242 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.494182 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-kube-api-access-djmws" pod "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" (UID: "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5"). InnerVolumeSpecName "kube-api-access-djmws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 16:00:15.494519 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.494494 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-podinfo" pod "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" (UID: "85ad1f5d-e58c-4949-839e-4f2fedc6f7c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 16:00:15.528794 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.528765 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 16:00:15.535234 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.531262 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 16:00:15.584576 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.584478 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-workload-socket\") pod \"91ed8818-5236-42e2-92f5-74e29232e3e5\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " Oct 30 16:00:15.584576 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.584531 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ab3b174-5788-448b-b85f-49c879be5489-istio-token\" (UniqueName: \"kubernetes.io/projected/2ab3b174-5788-448b-b85f-49c879be5489-istio-token\") pod \"2ab3b174-5788-448b-b85f-49c879be5489\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " Oct 30 16:00:15.584576 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.584568 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ab3b174-5788-448b-b85f-49c879be5489-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ab3b174-5788-448b-b85f-49c879be5489-istiod-ca-cert\") pod \"2ab3b174-5788-448b-b85f-49c879be5489\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " Oct 30 16:00:15.584961 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.584598 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-istio-envoy\") pod \"2ab3b174-5788-448b-b85f-49c879be5489\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " Oct 30 16:00:15.584961 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.584641 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-workload-certs\") pod \"2ab3b174-5788-448b-b85f-49c879be5489\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " Oct 30 16:00:15.584961 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.584703 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-data\") pod \"91ed8818-5236-42e2-92f5-74e29232e3e5\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " Oct 30 16:00:15.584961 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.584737 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-credential-socket\") pod \"2ab3b174-5788-448b-b85f-49c879be5489\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " Oct 30 16:00:15.584961 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.584767 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-credential-socket\") pod \"91ed8818-5236-42e2-92f5-74e29232e3e5\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " Oct 30 16:00:15.584961 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.584796 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-envoy\") pod \"91ed8818-5236-42e2-92f5-74e29232e3e5\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " Oct 30 16:00:15.584961 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.584836 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91ed8818-5236-42e2-92f5-74e29232e3e5-kube-api-access-fmw7h\" (UniqueName: \"kubernetes.io/projected/91ed8818-5236-42e2-92f5-74e29232e3e5-kube-api-access-fmw7h\") pod \"91ed8818-5236-42e2-92f5-74e29232e3e5\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " Oct 30 16:00:15.584961 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.584882 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-workload-socket\") pod \"2ab3b174-5788-448b-b85f-49c879be5489\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " Oct 30 16:00:15.584961 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.584922 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-istio-data\") pod \"2ab3b174-5788-448b-b85f-49c879be5489\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " Oct 30 16:00:15.584961 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.584948 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91ed8818-5236-42e2-92f5-74e29232e3e5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91ed8818-5236-42e2-92f5-74e29232e3e5-istiod-ca-cert\") pod \"91ed8818-5236-42e2-92f5-74e29232e3e5\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " Oct 30 16:00:15.585559 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.584978 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ab3b174-5788-448b-b85f-49c879be5489-kube-api-access-zz6q6\" (UniqueName: \"kubernetes.io/projected/2ab3b174-5788-448b-b85f-49c879be5489-kube-api-access-zz6q6\") pod \"2ab3b174-5788-448b-b85f-49c879be5489\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " Oct 30 16:00:15.585559 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.585023 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-podinfo\") pod \"91ed8818-5236-42e2-92f5-74e29232e3e5\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " Oct 30 16:00:15.585559 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.585057 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2ab3b174-5788-448b-b85f-49c879be5489-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ab3b174-5788-448b-b85f-49c879be5489-istio-podinfo\") pod \"2ab3b174-5788-448b-b85f-49c879be5489\" (UID: \"2ab3b174-5788-448b-b85f-49c879be5489\") " Oct 30 16:00:15.585559 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.585127 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-token\" (UniqueName: \"kubernetes.io/projected/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-token\") pod \"91ed8818-5236-42e2-92f5-74e29232e3e5\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " Oct 30 16:00:15.585559 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.585166 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-workload-certs\") pod \"91ed8818-5236-42e2-92f5-74e29232e3e5\" (UID: \"91ed8818-5236-42e2-92f5-74e29232e3e5\") " Oct 30 16:00:15.585559 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.585526 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djmws\" (UniqueName: \"kubernetes.io/projected/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-kube-api-access-djmws\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.585559 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.585546 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.585559 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.585560 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.585989 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.585574 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.585989 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.585587 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.585989 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.585599 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.585989 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.585611 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.585989 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.585624 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.585989 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.585637 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.585989 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.585887 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-workload-certs" pod "91ed8818-5236-42e2-92f5-74e29232e3e5" (UID: "91ed8818-5236-42e2-92f5-74e29232e3e5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.586383 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.586100 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-workload-socket" pod "91ed8818-5236-42e2-92f5-74e29232e3e5" (UID: "91ed8818-5236-42e2-92f5-74e29232e3e5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.587574 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.587543 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-workload-socket" pod "2ab3b174-5788-448b-b85f-49c879be5489" (UID: "2ab3b174-5788-448b-b85f-49c879be5489"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.587740 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.587721 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-workload-certs" pod "2ab3b174-5788-448b-b85f-49c879be5489" (UID: "2ab3b174-5788-448b-b85f-49c879be5489"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.588160 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.588132 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-istio-data" pod "2ab3b174-5788-448b-b85f-49c879be5489" (UID: "2ab3b174-5788-448b-b85f-49c879be5489"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.588479 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.588448 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ab3b174-5788-448b-b85f-49c879be5489-istiod-ca-cert" pod "2ab3b174-5788-448b-b85f-49c879be5489" (UID: "2ab3b174-5788-448b-b85f-49c879be5489"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 16:00:15.588767 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.588746 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-credential-socket" pod "91ed8818-5236-42e2-92f5-74e29232e3e5" (UID: "91ed8818-5236-42e2-92f5-74e29232e3e5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.588928 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.588910 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-credential-socket" pod "2ab3b174-5788-448b-b85f-49c879be5489" (UID: "2ab3b174-5788-448b-b85f-49c879be5489"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.589214 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.589170 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-data" pod "91ed8818-5236-42e2-92f5-74e29232e3e5" (UID: "91ed8818-5236-42e2-92f5-74e29232e3e5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.589653 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.589626 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ab3b174-5788-448b-b85f-49c879be5489-istio-token" pod "2ab3b174-5788-448b-b85f-49c879be5489" (UID: "2ab3b174-5788-448b-b85f-49c879be5489"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 16:00:15.589889 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.589865 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91ed8818-5236-42e2-92f5-74e29232e3e5-istiod-ca-cert" pod "91ed8818-5236-42e2-92f5-74e29232e3e5" (UID: "91ed8818-5236-42e2-92f5-74e29232e3e5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 16:00:15.590582 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.590534 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91ed8818-5236-42e2-92f5-74e29232e3e5-kube-api-access-fmw7h" pod "91ed8818-5236-42e2-92f5-74e29232e3e5" (UID: "91ed8818-5236-42e2-92f5-74e29232e3e5"). InnerVolumeSpecName "kube-api-access-fmw7h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 16:00:15.590679 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.590592 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ab3b174-5788-448b-b85f-49c879be5489-kube-api-access-zz6q6" pod "2ab3b174-5788-448b-b85f-49c879be5489" (UID: "2ab3b174-5788-448b-b85f-49c879be5489"). InnerVolumeSpecName "kube-api-access-zz6q6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 16:00:15.591293 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.591255 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ab3b174-5788-448b-b85f-49c879be5489-istio-podinfo" pod "2ab3b174-5788-448b-b85f-49c879be5489" (UID: "2ab3b174-5788-448b-b85f-49c879be5489"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 16:00:15.591627 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.591603 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-podinfo" pod "91ed8818-5236-42e2-92f5-74e29232e3e5" (UID: "91ed8818-5236-42e2-92f5-74e29232e3e5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 16:00:15.592513 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.592488 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-token" pod "91ed8818-5236-42e2-92f5-74e29232e3e5" (UID: "91ed8818-5236-42e2-92f5-74e29232e3e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 16:00:15.592780 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.592760 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-envoy" pod "91ed8818-5236-42e2-92f5-74e29232e3e5" (UID: "91ed8818-5236-42e2-92f5-74e29232e3e5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.594309 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.594171 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-istio-envoy" pod "2ab3b174-5788-448b-b85f-49c879be5489" (UID: "2ab3b174-5788-448b-b85f-49c879be5489"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 16:00:15.686691 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686659 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.686691 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686686 5013 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ab3b174-5788-448b-b85f-49c879be5489-istio-podinfo\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.686691 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686695 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.686691 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686703 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.687044 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686712 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.687044 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686720 5013 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ab3b174-5788-448b-b85f-49c879be5489-istio-token\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.687044 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686727 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ab3b174-5788-448b-b85f-49c879be5489-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.687044 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686735 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.687044 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686742 5013 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-workload-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.687044 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686749 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.687044 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686756 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.687044 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686769 5013 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-credential-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.687044 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686780 5013 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91ed8818-5236-42e2-92f5-74e29232e3e5-istio-envoy\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.687044 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686788 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmw7h\" (UniqueName: \"kubernetes.io/projected/91ed8818-5236-42e2-92f5-74e29232e3e5-kube-api-access-fmw7h\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.687044 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686797 5013 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-workload-socket\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.687044 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686804 5013 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ab3b174-5788-448b-b85f-49c879be5489-istio-data\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.687044 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686812 5013 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91ed8818-5236-42e2-92f5-74e29232e3e5-istiod-ca-cert\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:15.687044 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:15.686819 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zz6q6\" (UniqueName: \"kubernetes.io/projected/2ab3b174-5788-448b-b85f-49c879be5489-kube-api-access-zz6q6\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:00:16.035002 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.034972 5013 generic.go:356] "Generic (PLEG): container finished" podID="2ab3b174-5788-448b-b85f-49c879be5489" containerID="d0e2f475327b3a06ac5e995f35637b3f628c27c497d008b8364f52589b405118" exitCode=0 Oct 30 16:00:16.035117 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.035021 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" Oct 30 16:00:16.035117 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.035050 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" event={"ID":"2ab3b174-5788-448b-b85f-49c879be5489","Type":"ContainerDied","Data":"d0e2f475327b3a06ac5e995f35637b3f628c27c497d008b8364f52589b405118"} Oct 30 16:00:16.035117 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.035100 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2" event={"ID":"2ab3b174-5788-448b-b85f-49c879be5489","Type":"ContainerDied","Data":"c962a025c5b260f7beb884da672be867eccac700aef98623b02378e0fb2926e7"} Oct 30 16:00:16.035336 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.035124 5013 scope.go:122] "RemoveContainer" containerID="d0e2f475327b3a06ac5e995f35637b3f628c27c497d008b8364f52589b405118" Oct 30 16:00:16.036636 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.036603 5013 generic.go:356] "Generic (PLEG): container finished" podID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" containerID="6a81096f9204bfe6a66727dd87a3f77c301b728338307cbba351decf5c4d2913" exitCode=0 Oct 30 16:00:16.036712 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.036661 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" event={"ID":"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5","Type":"ContainerDied","Data":"6a81096f9204bfe6a66727dd87a3f77c301b728338307cbba351decf5c4d2913"} Oct 30 16:00:16.036712 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.036690 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" event={"ID":"85ad1f5d-e58c-4949-839e-4f2fedc6f7c5","Type":"ContainerDied","Data":"a22b76f21806243db84661a387895b44cbddb0979f0627e98a287ff6464e5970"} Oct 30 16:00:16.036825 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.036757 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z" Oct 30 16:00:16.038498 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.038476 5013 generic.go:356] "Generic (PLEG): container finished" podID="91ed8818-5236-42e2-92f5-74e29232e3e5" containerID="8761b289dee89aa1f9575cbf1c5e82dc33d096463fe3d8abd63359749176ab90" exitCode=0 Oct 30 16:00:16.038610 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.038563 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" event={"ID":"91ed8818-5236-42e2-92f5-74e29232e3e5","Type":"ContainerDied","Data":"8761b289dee89aa1f9575cbf1c5e82dc33d096463fe3d8abd63359749176ab90"} Oct 30 16:00:16.038681 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.038613 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" event={"ID":"91ed8818-5236-42e2-92f5-74e29232e3e5","Type":"ContainerDied","Data":"7b59e537fd8adfd6822797108cda446b4c1d17b8999597b6393c70599d79e58b"} Oct 30 16:00:16.038806 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.038783 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t" Oct 30 16:00:16.040047 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.040029 5013 generic.go:356] "Generic (PLEG): container finished" podID="5adc0947-063e-4c71-a2c6-d8395cc3b0d2" containerID="19110742045fe84ab2b11e12484af32385528c38e0148b5dcc58fe02a168da7f" exitCode=0 Oct 30 16:00:16.040159 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.040052 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5964675dcd-lswsg" event={"ID":"5adc0947-063e-4c71-a2c6-d8395cc3b0d2","Type":"ContainerDied","Data":"19110742045fe84ab2b11e12484af32385528c38e0148b5dcc58fe02a168da7f"} Oct 30 16:00:16.040159 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.040083 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5964675dcd-lswsg" event={"ID":"5adc0947-063e-4c71-a2c6-d8395cc3b0d2","Type":"ContainerDied","Data":"58a322e28366a061dbda8acb7eb9a15d00aae6259a1e22788a8590d5b5cd0a1a"} Oct 30 16:00:16.040159 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.040156 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5964675dcd-lswsg" Oct 30 16:00:16.046424 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.046404 5013 scope.go:122] "RemoveContainer" containerID="d0e2f475327b3a06ac5e995f35637b3f628c27c497d008b8364f52589b405118" Oct 30 16:00:16.046706 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:00:16.046677 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0e2f475327b3a06ac5e995f35637b3f628c27c497d008b8364f52589b405118\": container with ID starting with d0e2f475327b3a06ac5e995f35637b3f628c27c497d008b8364f52589b405118 not found: ID does not exist" containerID="d0e2f475327b3a06ac5e995f35637b3f628c27c497d008b8364f52589b405118" Oct 30 16:00:16.046810 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.046703 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e2f475327b3a06ac5e995f35637b3f628c27c497d008b8364f52589b405118"} err="failed to get container status \"d0e2f475327b3a06ac5e995f35637b3f628c27c497d008b8364f52589b405118\": rpc error: code = NotFound desc = could not find container \"d0e2f475327b3a06ac5e995f35637b3f628c27c497d008b8364f52589b405118\": container with ID starting with d0e2f475327b3a06ac5e995f35637b3f628c27c497d008b8364f52589b405118 not found: ID does not exist" Oct 30 16:00:16.046810 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.046719 5013 scope.go:122] "RemoveContainer" containerID="6a81096f9204bfe6a66727dd87a3f77c301b728338307cbba351decf5c4d2913" Oct 30 16:00:16.056416 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.056400 5013 scope.go:122] "RemoveContainer" containerID="6a81096f9204bfe6a66727dd87a3f77c301b728338307cbba351decf5c4d2913" Oct 30 16:00:16.056754 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:00:16.056737 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a81096f9204bfe6a66727dd87a3f77c301b728338307cbba351decf5c4d2913\": container with ID starting with 6a81096f9204bfe6a66727dd87a3f77c301b728338307cbba351decf5c4d2913 not found: ID does not exist" containerID="6a81096f9204bfe6a66727dd87a3f77c301b728338307cbba351decf5c4d2913" Oct 30 16:00:16.056841 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.056764 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a81096f9204bfe6a66727dd87a3f77c301b728338307cbba351decf5c4d2913"} err="failed to get container status \"6a81096f9204bfe6a66727dd87a3f77c301b728338307cbba351decf5c4d2913\": rpc error: code = NotFound desc = could not find container \"6a81096f9204bfe6a66727dd87a3f77c301b728338307cbba351decf5c4d2913\": container with ID starting with 6a81096f9204bfe6a66727dd87a3f77c301b728338307cbba351decf5c4d2913 not found: ID does not exist" Oct 30 16:00:16.056841 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.056780 5013 scope.go:122] "RemoveContainer" containerID="8761b289dee89aa1f9575cbf1c5e82dc33d096463fe3d8abd63359749176ab90" Oct 30 16:00:16.082499 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.082472 5013 scope.go:122] "RemoveContainer" containerID="8761b289dee89aa1f9575cbf1c5e82dc33d096463fe3d8abd63359749176ab90" Oct 30 16:00:16.082759 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:00:16.082742 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8761b289dee89aa1f9575cbf1c5e82dc33d096463fe3d8abd63359749176ab90\": container with ID starting with 8761b289dee89aa1f9575cbf1c5e82dc33d096463fe3d8abd63359749176ab90 not found: ID does not exist" containerID="8761b289dee89aa1f9575cbf1c5e82dc33d096463fe3d8abd63359749176ab90" Oct 30 16:00:16.082822 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.082765 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8761b289dee89aa1f9575cbf1c5e82dc33d096463fe3d8abd63359749176ab90"} err="failed to get container status \"8761b289dee89aa1f9575cbf1c5e82dc33d096463fe3d8abd63359749176ab90\": rpc error: code = NotFound desc = could not find container \"8761b289dee89aa1f9575cbf1c5e82dc33d096463fe3d8abd63359749176ab90\": container with ID starting with 8761b289dee89aa1f9575cbf1c5e82dc33d096463fe3d8abd63359749176ab90 not found: ID does not exist" Oct 30 16:00:16.082822 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.082785 5013 scope.go:122] "RemoveContainer" containerID="19110742045fe84ab2b11e12484af32385528c38e0148b5dcc58fe02a168da7f" Oct 30 16:00:16.093365 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.093343 5013 scope.go:122] "RemoveContainer" containerID="19110742045fe84ab2b11e12484af32385528c38e0148b5dcc58fe02a168da7f" Oct 30 16:00:16.093636 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:00:16.093617 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19110742045fe84ab2b11e12484af32385528c38e0148b5dcc58fe02a168da7f\": container with ID starting with 19110742045fe84ab2b11e12484af32385528c38e0148b5dcc58fe02a168da7f not found: ID does not exist" containerID="19110742045fe84ab2b11e12484af32385528c38e0148b5dcc58fe02a168da7f" Oct 30 16:00:16.093709 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.093641 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19110742045fe84ab2b11e12484af32385528c38e0148b5dcc58fe02a168da7f"} err="failed to get container status \"19110742045fe84ab2b11e12484af32385528c38e0148b5dcc58fe02a168da7f\": rpc error: code = NotFound desc = could not find container \"19110742045fe84ab2b11e12484af32385528c38e0148b5dcc58fe02a168da7f\": container with ID starting with 19110742045fe84ab2b11e12484af32385528c38e0148b5dcc58fe02a168da7f not found: ID does not exist" Oct 30 16:00:16.099958 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.099934 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t"] Oct 30 16:00:16.120910 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.120884 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-hxz5t"] Oct 30 16:00:16.182283 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.182262 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z"] Oct 30 16:00:16.201090 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.201064 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-vcp7z"] Oct 30 16:00:16.247858 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.247828 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5964675dcd-lswsg"] Oct 30 16:00:16.251010 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.250975 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5964675dcd-lswsg"] Oct 30 16:00:16.297091 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.297017 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2"] Oct 30 16:00:16.300661 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:16.300640 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-r7gk2"] Oct 30 16:00:17.509299 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:17.509151 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:17.878282 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:17.874748 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:18.008915 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:18.008876 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ab3b174-5788-448b-b85f-49c879be5489" path="/var/lib/kubelet/pods/2ab3b174-5788-448b-b85f-49c879be5489/volumes" Oct 30 16:00:18.009371 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:18.009351 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5adc0947-063e-4c71-a2c6-d8395cc3b0d2" path="/var/lib/kubelet/pods/5adc0947-063e-4c71-a2c6-d8395cc3b0d2/volumes" Oct 30 16:00:18.009818 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:18.009797 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" path="/var/lib/kubelet/pods/85ad1f5d-e58c-4949-839e-4f2fedc6f7c5/volumes" Oct 30 16:00:18.010539 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:18.010518 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91ed8818-5236-42e2-92f5-74e29232e3e5" path="/var/lib/kubelet/pods/91ed8818-5236-42e2-92f5-74e29232e3e5/volumes" Oct 30 16:00:18.206632 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:18.206543 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:18.514479 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:18.514396 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:20.813925 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:20.813877 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:26.642592 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:26.642557 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:26.975754 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:00:26.975671 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:00:40.220319 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:40.220283 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:40.404921 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:40.404887 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:51.516256 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:51.516225 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:52.984080 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:52.984032 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:53.173993 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:53.173964 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:53.364171 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:53.364094 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:54.228110 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228078 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz"] Oct 30 16:00:54.228515 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228471 5013 cpu_manager.go:414] "containerMap: removing container" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" Oct 30 16:00:54.228515 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228483 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" Oct 30 16:00:54.228515 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228495 5013 cpu_manager.go:414] "containerMap: removing container" podUID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" containerName="istio-proxy" Oct 30 16:00:54.228515 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228500 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" containerName="istio-proxy" Oct 30 16:00:54.228515 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228515 5013 cpu_manager.go:414] "containerMap: removing container" podUID="f52a418d-1057-4580-8728-3fe24c4d308d" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228520 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f52a418d-1057-4580-8728-3fe24c4d308d" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228532 5013 cpu_manager.go:414] "containerMap: removing container" podUID="b0f87920-2518-4aa2-9db4-5959b7e71a54" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228536 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0f87920-2518-4aa2-9db4-5959b7e71a54" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228542 5013 cpu_manager.go:414] "containerMap: removing container" podUID="5adc0947-063e-4c71-a2c6-d8395cc3b0d2" containerName="manager" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228546 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5adc0947-063e-4c71-a2c6-d8395cc3b0d2" containerName="manager" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228555 5013 cpu_manager.go:414] "containerMap: removing container" podUID="4e6a88f2-c53c-4346-81b6-68ea50a6528a" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228559 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e6a88f2-c53c-4346-81b6-68ea50a6528a" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228565 5013 cpu_manager.go:414] "containerMap: removing container" podUID="91ed8818-5236-42e2-92f5-74e29232e3e5" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228570 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91ed8818-5236-42e2-92f5-74e29232e3e5" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228578 5013 cpu_manager.go:414] "containerMap: removing container" podUID="1f037ba0-aee4-4f9e-8b48-a3e7c1520671" containerName="maas-api" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228583 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f037ba0-aee4-4f9e-8b48-a3e7c1520671" containerName="maas-api" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228594 5013 cpu_manager.go:414] "containerMap: removing container" podUID="2ab3b174-5788-448b-b85f-49c879be5489" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228598 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ab3b174-5788-448b-b85f-49c879be5489" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228607 5013 cpu_manager.go:414] "containerMap: removing container" podUID="ea1e0986-aa76-4df4-85ac-cf104e42fb7e" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228611 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea1e0986-aa76-4df4-85ac-cf104e42fb7e" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228616 5013 cpu_manager.go:414] "containerMap: removing container" podUID="ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228621 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228626 5013 cpu_manager.go:414] "containerMap: removing container" podUID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228644 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228651 5013 cpu_manager.go:414] "containerMap: removing container" podUID="b477fa47-61b7-4232-a375-6ca81dfec7cf" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228656 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b477fa47-61b7-4232-a375-6ca81dfec7cf" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228663 5013 cpu_manager.go:414] "containerMap: removing container" podUID="04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" containerName="istio-proxy" Oct 30 16:00:54.228672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228667 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" containerName="istio-proxy" Oct 30 16:00:54.229560 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228720 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea1e0986-aa76-4df4-85ac-cf104e42fb7e" containerName="istio-proxy" Oct 30 16:00:54.229560 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228728 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef8ef827-bafc-4f76-96d3-4a5371e8bdaa" containerName="istio-proxy" Oct 30 16:00:54.229560 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228732 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="a06cb93e-6948-474e-bc0c-6601b10e1e3d" containerName="istio-proxy" Oct 30 16:00:54.229560 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228738 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="04a27f82-3d2b-4bc3-9e2c-3fd0f54da2f9" containerName="istio-proxy" Oct 30 16:00:54.229560 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228744 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="c636d9c4-4262-46ba-9fd4-64388f70a905" containerName="main" Oct 30 16:00:54.229560 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228748 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="85ad1f5d-e58c-4949-839e-4f2fedc6f7c5" containerName="istio-proxy" Oct 30 16:00:54.229560 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228753 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="f52a418d-1057-4580-8728-3fe24c4d308d" containerName="istio-proxy" Oct 30 16:00:54.229560 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228759 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0f87920-2518-4aa2-9db4-5959b7e71a54" containerName="istio-proxy" Oct 30 16:00:54.229560 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228764 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="5adc0947-063e-4c71-a2c6-d8395cc3b0d2" containerName="manager" Oct 30 16:00:54.229560 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228769 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e6a88f2-c53c-4346-81b6-68ea50a6528a" containerName="istio-proxy" Oct 30 16:00:54.229560 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228774 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="91ed8818-5236-42e2-92f5-74e29232e3e5" containerName="istio-proxy" Oct 30 16:00:54.229560 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228778 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f037ba0-aee4-4f9e-8b48-a3e7c1520671" containerName="maas-api" Oct 30 16:00:54.229560 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228784 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ab3b174-5788-448b-b85f-49c879be5489" containerName="istio-proxy" Oct 30 16:00:54.229560 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.228788 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="b477fa47-61b7-4232-a375-6ca81dfec7cf" containerName="istio-proxy" Oct 30 16:00:54.233094 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.233072 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.256832 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.256811 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz"] Oct 30 16:00:54.340557 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.340523 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.340697 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.340561 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c23430c-a96b-404c-8772-0fa74158186b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.340697 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.340591 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqtvk\" (UniqueName: \"kubernetes.io/projected/8c23430c-a96b-404c-8772-0fa74158186b-kube-api-access-sqtvk\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.340778 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.340741 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.340778 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.340766 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c23430c-a96b-404c-8772-0fa74158186b-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.340869 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.340796 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.340939 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.340922 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c23430c-a96b-404c-8772-0fa74158186b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.340992 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.340956 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.341053 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.341025 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.441943 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.441911 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c23430c-a96b-404c-8772-0fa74158186b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.442096 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.441949 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.442096 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.441990 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.442096 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.442023 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.442096 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.442047 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c23430c-a96b-404c-8772-0fa74158186b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.442096 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.442076 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqtvk\" (UniqueName: \"kubernetes.io/projected/8c23430c-a96b-404c-8772-0fa74158186b-kube-api-access-sqtvk\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.442436 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.442151 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.442436 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.442176 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c23430c-a96b-404c-8772-0fa74158186b-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.442436 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.442250 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.442436 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.442360 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.442668 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.442448 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.442668 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.442602 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.442668 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.442649 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.442775 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.442681 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c23430c-a96b-404c-8772-0fa74158186b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.444459 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.444439 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c23430c-a96b-404c-8772-0fa74158186b-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.444814 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.444793 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c23430c-a96b-404c-8772-0fa74158186b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.465166 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.465137 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c23430c-a96b-404c-8772-0fa74158186b-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.465345 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.465328 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqtvk\" (UniqueName: \"kubernetes.io/projected/8c23430c-a96b-404c-8772-0fa74158186b-kube-api-access-sqtvk\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gqsbz\" (UID: \"8c23430c-a96b-404c-8772-0fa74158186b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.546015 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.545965 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:54.859522 ip-10-0-129-40 kubenswrapper[5013]: W1030 16:00:54.859490 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c23430c_a96b_404c_8772_0fa74158186b.slice/crio-7c921a6dd9d36c1d7695d0374a9aaf16ff1e6346c2fd2970a812a7766ce8d4ab WatchSource:0}: Error finding container 7c921a6dd9d36c1d7695d0374a9aaf16ff1e6346c2fd2970a812a7766ce8d4ab: Status 404 returned error can't find the container with id 7c921a6dd9d36c1d7695d0374a9aaf16ff1e6346c2fd2970a812a7766ce8d4ab Oct 30 16:00:54.861443 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.861412 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 16:00:54.861535 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.861491 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 16:00:54.861608 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.861530 5013 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"} Oct 30 16:00:54.867301 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:54.867282 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz"] Oct 30 16:00:55.230769 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:55.230728 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" event={"ID":"8c23430c-a96b-404c-8772-0fa74158186b","Type":"ContainerStarted","Data":"a86cf26620fb17f9a22f67b08e00a877cd2bfeab4e0f28ebde3b0f70c14e60ea"} Oct 30 16:00:55.231281 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:55.230776 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" event={"ID":"8c23430c-a96b-404c-8772-0fa74158186b","Type":"ContainerStarted","Data":"7c921a6dd9d36c1d7695d0374a9aaf16ff1e6346c2fd2970a812a7766ce8d4ab"} Oct 30 16:00:55.291977 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:55.283743 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" podStartSLOduration=1.283723698 podStartE2EDuration="1.283723698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 16:00:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:00:55.277767097 +0000 UTC m=+1371.428664132" watchObservedRunningTime="2026-10-30 16:00:55.283723698 +0000 UTC m=+1371.434620730" Oct 30 16:00:55.546700 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:55.546608 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:00:55.548217 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:55.548171 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 16:00:55.548346 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:55.548262 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" podUID="8c23430c-a96b-404c-8772-0fa74158186b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 16:00:56.040424 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:56.040379 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:00:56.546966 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:56.546938 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 16:00:56.547365 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:56.546998 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" podUID="8c23430c-a96b-404c-8772-0fa74158186b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 16:00:56.999153 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:00:56.999116 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:00:57.547135 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:57.547100 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 16:00:57.547530 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:57.547154 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" podUID="8c23430c-a96b-404c-8772-0fa74158186b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 16:00:58.546640 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:58.546604 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 16:00:58.546818 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:58.546656 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" podUID="8c23430c-a96b-404c-8772-0fa74158186b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 16:00:59.546479 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:59.546441 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 16:00:59.546867 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:00:59.546493 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" podUID="8c23430c-a96b-404c-8772-0fa74158186b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 16:01:00.546932 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:00.546891 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 16:01:00.547439 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:00.546962 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" podUID="8c23430c-a96b-404c-8772-0fa74158186b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 16:01:01.546840 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:01.546808 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 16:01:01.547035 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:01.546865 5013 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" podUID="8c23430c-a96b-404c-8772-0fa74158186b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 16:01:02.952586 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:02.952550 5013 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:01:02.953065 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:02.952957 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:01:02.953786 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:02.953766 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" Oct 30 16:01:15.579722 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:15.579641 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:01:22.063674 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:22.063640 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:01:27.039770 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:01:27.039703 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:01:33.514910 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:33.514875 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:01:46.360790 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:46.360758 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:01:49.001220 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:49.000928 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 16:01:49.001220 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:49.001000 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" podUID="8c23430c-a96b-404c-8772-0fa74158186b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 16:01:49.193329 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:49.193293 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 16:01:49.193619 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:01:49.193591 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-bl2mn" podUID="3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 16:01:56.975976 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:01:56.975938 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:02:02.257648 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:02:02.257580 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:02:02.424251 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:02:02.424219 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:02:26.976256 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:02:26.976220 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:02:56.355489 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:02:56.355452 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:02:56.980375 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:02:56.980327 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:03:04.166189 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:03:04.166162 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 16:03:04.170588 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:03:04.170563 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 16:03:18.986826 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:03:18.986777 5013 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 16:03:18.987439 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:03:18.987387 5013 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gqsbz" podUID="8c23430c-a96b-404c-8772-0fa74158186b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 16:03:26.975932 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:03:26.975895 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:03:42.431218 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:03:42.431023 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:03:42.628224 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:03:42.628173 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:03:56.411967 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:03:56.411931 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:03:56.976843 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:03:56.976806 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:04:27.003979 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:04:27.003789 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:04:56.534258 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:04:56.534164 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:04:56.975344 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:04:56.975310 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:05:19.515973 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:19.515938 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:20.077328 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:20.077290 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:20.310432 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:20.310394 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:23.554757 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:23.554717 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:23.772354 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:23.772325 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:25.114486 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:25.114452 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:25.299604 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:25.299534 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:25.479380 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:25.479347 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:26.866827 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:26.866789 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:26.975458 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:05:26.975427 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:05:28.602383 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:28.602344 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:30.435331 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:30.435288 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:32.180690 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:32.180636 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:33.890388 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:33.890326 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:35.690167 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:35.690134 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:37.613187 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:37.613150 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:38.557845 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:38.557812 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:39.223028 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:39.222054 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:39.392616 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:39.392588 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:39.556586 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:39.556498 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:41.193955 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:41.193912 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:41.438434 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:41.438394 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:43.096156 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:43.096127 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:44.922868 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:44.922790 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:46.775037 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:46.775001 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:48.603494 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:48.603463 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:50.328658 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:50.328622 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:52.160133 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:52.160107 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:53.864484 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:53.864454 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:55.736824 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:55.736785 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:56.653146 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:56.653107 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:56.975356 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:05:56.975280 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:05:57.625749 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:57.625717 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:05:59.474355 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:05:59.474325 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:01.114032 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:01.114002 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:02.847726 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:02.847690 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:04.661533 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:04.661500 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:07.598401 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:07.598370 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:08.282293 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:08.282258 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:09.909110 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:09.909077 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:11.732406 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:11.732373 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:13.428026 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:13.427991 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:15.186940 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:15.186896 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:15.858368 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:15.858334 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:17.270450 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:17.270414 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:19.169617 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:19.169583 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:20.865860 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:20.865830 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:21.036611 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:21.036582 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:22.683337 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:22.683309 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:24.484491 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:24.484422 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:26.241368 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:26.241333 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:26.648875 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:26.648848 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:26.817426 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:26.817383 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:26.976657 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:06:26.976582 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:06:28.073412 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:28.073375 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:29.655463 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:29.655434 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:30.271763 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:30.271732 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:30.510103 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:30.510070 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:30.977522 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:30.977485 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:37.610350 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:37.610318 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:37.836740 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:37.836708 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:38.018659 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:38.018624 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:49.251745 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:49.251717 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:56.781710 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:56.781678 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:56.975975 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:06:56.975942 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:06:57.992838 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:57.992808 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:06:58.765494 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:06:58.765454 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:07:05.055400 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:07:05.055371 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:07:20.960823 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:07:20.960770 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:07:26.976172 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:07:26.976138 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:07:56.913562 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:07:56.913528 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:07:56.975593 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:07:56.975559 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:08:04.195351 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:08:04.195314 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 16:08:04.200609 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:08:04.200587 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 16:08:27.036650 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:08:27.036611 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:08:56.975724 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:08:56.975686 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:08:57.043146 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:08:57.043118 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:08:58.704093 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:08:58.704051 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:01.599099 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:01.599057 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:01.783124 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:01.783096 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:03.236928 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:03.236888 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:03.408255 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:03.408224 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:03.568588 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:03.568509 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:05.164996 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:05.164963 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:06.939682 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:06.939649 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:08.730828 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:08.730791 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:10.352240 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:10.352175 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:12.150553 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:12.150502 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:14.007952 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:14.007924 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:15.760876 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:15.760843 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:16.536776 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:16.536733 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:17.185765 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:17.185726 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:17.425555 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:17.425523 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:17.603792 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:17.603705 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:19.554798 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:19.554765 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:19.735568 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:19.735533 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:21.359372 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:21.359339 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:23.202527 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:23.202493 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:24.849583 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:24.849550 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:26.566130 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:26.566095 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:26.976575 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:09:26.976547 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:09:28.441519 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:28.441489 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:30.122444 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:30.122400 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:32.226941 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:32.226905 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:33.613511 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:33.613474 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:35.438750 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:35.438702 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:37.127507 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:37.127477 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:38.873139 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:38.873105 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:40.499566 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:40.499518 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:42.233215 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:42.233149 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:43.970626 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:43.970591 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:45.620129 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:45.620098 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:47.313291 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:47.313258 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:48.978721 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:48.978684 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:49.447245 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:49.447190 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:50.768367 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:50.768335 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:50.932229 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:50.932179 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:52.674038 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:52.674008 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:54.546872 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:54.546838 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:54.709574 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:54.709539 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:56.301158 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:56.301125 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:56.976329 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:09:56.976289 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:09:57.151317 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:57.151283 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:57.307735 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:57.307669 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:58.004170 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:58.004139 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:09:59.823930 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:09:59.823897 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:00.189811 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:00.189776 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:00.413393 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:00.413341 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:01.506799 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:01.506764 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:02.966416 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:02.966387 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:03.600232 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:03.600176 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:03.765376 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:03.765343 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:04.257730 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:04.257684 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:09.862241 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:09.862187 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:10.780487 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:10.780450 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:10.942365 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:10.942335 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:11.414585 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:11.414525 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:15.346718 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:15.346628 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:23.843876 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:23.843846 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:24.013343 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:24.013315 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:26.976646 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:10:26.976607 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:10:31.701221 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:31.701162 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:32.288800 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:32.288762 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:33.285088 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:33.285053 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:36.252952 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:36.252916 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:36.412733 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:36.412698 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:38.055107 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:38.055072 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:10:56.975560 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:10:56.975521 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:10:57.285186 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:10:57.285072 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:11:02.938122 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:02.938070 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:11:05.961930 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:05.961855 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:11:08.775677 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:08.775644 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:11:08.937085 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:08.937050 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:11:27.033703 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:11:27.033635 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:11:28.200369 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:28.200337 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:11:28.370495 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:28.370466 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:11:53.213643 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.213569 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-fmb6s"] Oct 30 16:11:53.217246 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.217229 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:53.245390 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.245358 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-fmb6s"] Oct 30 16:11:53.357828 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.357786 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f05e504c-8b40-44de-ab8b-445f0ac4d80d-openshift-service-ca\") pod \"maas-api-595cc648c-fmb6s\" (UID: \"f05e504c-8b40-44de-ab8b-445f0ac4d80d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:53.357828 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.357839 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljfvj\" (UniqueName: \"kubernetes.io/projected/f05e504c-8b40-44de-ab8b-445f0ac4d80d-kube-api-access-ljfvj\") pod \"maas-api-595cc648c-fmb6s\" (UID: \"f05e504c-8b40-44de-ab8b-445f0ac4d80d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:53.358031 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.357970 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f05e504c-8b40-44de-ab8b-445f0ac4d80d-metrics-certs\") pod \"maas-api-595cc648c-fmb6s\" (UID: \"f05e504c-8b40-44de-ab8b-445f0ac4d80d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:53.358073 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.358040 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f05e504c-8b40-44de-ab8b-445f0ac4d80d-maas-api-tls\") pod \"maas-api-595cc648c-fmb6s\" (UID: \"f05e504c-8b40-44de-ab8b-445f0ac4d80d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:53.458885 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.458857 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f05e504c-8b40-44de-ab8b-445f0ac4d80d-metrics-certs\") pod \"maas-api-595cc648c-fmb6s\" (UID: \"f05e504c-8b40-44de-ab8b-445f0ac4d80d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:53.459075 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.458919 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f05e504c-8b40-44de-ab8b-445f0ac4d80d-maas-api-tls\") pod \"maas-api-595cc648c-fmb6s\" (UID: \"f05e504c-8b40-44de-ab8b-445f0ac4d80d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:53.459075 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.458979 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f05e504c-8b40-44de-ab8b-445f0ac4d80d-openshift-service-ca\") pod \"maas-api-595cc648c-fmb6s\" (UID: \"f05e504c-8b40-44de-ab8b-445f0ac4d80d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:53.459075 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.459017 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljfvj\" (UniqueName: \"kubernetes.io/projected/f05e504c-8b40-44de-ab8b-445f0ac4d80d-kube-api-access-ljfvj\") pod \"maas-api-595cc648c-fmb6s\" (UID: \"f05e504c-8b40-44de-ab8b-445f0ac4d80d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:53.459706 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.459656 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f05e504c-8b40-44de-ab8b-445f0ac4d80d-openshift-service-ca\") pod \"maas-api-595cc648c-fmb6s\" (UID: \"f05e504c-8b40-44de-ab8b-445f0ac4d80d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:53.461414 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.461390 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f05e504c-8b40-44de-ab8b-445f0ac4d80d-maas-api-tls\") pod \"maas-api-595cc648c-fmb6s\" (UID: \"f05e504c-8b40-44de-ab8b-445f0ac4d80d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:53.461913 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.461892 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f05e504c-8b40-44de-ab8b-445f0ac4d80d-metrics-certs\") pod \"maas-api-595cc648c-fmb6s\" (UID: \"f05e504c-8b40-44de-ab8b-445f0ac4d80d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:53.482876 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.482794 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljfvj\" (UniqueName: \"kubernetes.io/projected/f05e504c-8b40-44de-ab8b-445f0ac4d80d-kube-api-access-ljfvj\") pod \"maas-api-595cc648c-fmb6s\" (UID: \"f05e504c-8b40-44de-ab8b-445f0ac4d80d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:53.530490 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.530462 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:53.832707 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.832685 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-fmb6s"] Oct 30 16:11:53.834735 ip-10-0-129-40 kubenswrapper[5013]: W1030 16:11:53.834706 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf05e504c_8b40_44de_ab8b_445f0ac4d80d.slice/crio-9c8d814d96b915f4379a27283a54820b7752826d53e41ce56bd7e7e4eca282df WatchSource:0}: Error finding container 9c8d814d96b915f4379a27283a54820b7752826d53e41ce56bd7e7e4eca282df: Status 404 returned error can't find the container with id 9c8d814d96b915f4379a27283a54820b7752826d53e41ce56bd7e7e4eca282df Oct 30 16:11:53.836049 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:53.836031 5013 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 16:11:54.584362 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:54.584329 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" event={"ID":"f05e504c-8b40-44de-ab8b-445f0ac4d80d","Type":"ContainerStarted","Data":"937784115dc9e2222c90b593ff276c6fd65db07b3549b3baee62bc2bd3e8db97"} Oct 30 16:11:54.584831 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:54.584376 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" event={"ID":"f05e504c-8b40-44de-ab8b-445f0ac4d80d","Type":"ContainerStarted","Data":"9c8d814d96b915f4379a27283a54820b7752826d53e41ce56bd7e7e4eca282df"} Oct 30 16:11:54.584831 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:54.584405 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:11:54.629012 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:54.628969 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" podStartSLOduration=1.34110981 podStartE2EDuration="1.628956957s" totalImagesPullingTime="287.847147ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 16:11:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:11:54.627123958 +0000 UTC m=+2030.778021000" watchObservedRunningTime="2026-10-30 16:11:54.628956957 +0000 UTC m=+2030.779853987" Oct 30 16:11:57.051473 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:11:57.051421 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:11:57.775786 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:11:57.775757 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:12:00.601328 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:00.601294 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-fmb6s" Oct 30 16:12:00.690663 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:00.690635 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-258kv"] Oct 30 16:12:00.690870 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:00.690830 5013 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" podUID="8d62df41-e0d5-4376-912f-f61f8d1fbd2d" containerName="maas-api" containerID="cri-o://b3de0a1c1e760905faa28a836fd2efa3b0611a33069b8189e30d6a53945bd793" gracePeriod=30 Oct 30 16:12:01.940621 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:01.940599 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 16:12:01.987725 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:01.987700 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-openshift-service-ca\") pod \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " Oct 30 16:12:01.987886 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:01.987735 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-kube-api-access-qfsm5\" (UniqueName: \"kubernetes.io/projected/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-kube-api-access-qfsm5\") pod \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " Oct 30 16:12:01.987886 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:01.987761 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-maas-api-tls\") pod \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " Oct 30 16:12:01.987886 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:01.987837 5013 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-metrics-certs\") pod \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\" (UID: \"8d62df41-e0d5-4376-912f-f61f8d1fbd2d\") " Oct 30 16:12:01.988158 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:01.988131 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-openshift-service-ca" pod "8d62df41-e0d5-4376-912f-f61f8d1fbd2d" (UID: "8d62df41-e0d5-4376-912f-f61f8d1fbd2d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 16:12:01.990410 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:01.990382 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-kube-api-access-qfsm5" pod "8d62df41-e0d5-4376-912f-f61f8d1fbd2d" (UID: "8d62df41-e0d5-4376-912f-f61f8d1fbd2d"). InnerVolumeSpecName "kube-api-access-qfsm5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 16:12:01.990520 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:01.990439 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-metrics-certs" pod "8d62df41-e0d5-4376-912f-f61f8d1fbd2d" (UID: "8d62df41-e0d5-4376-912f-f61f8d1fbd2d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 16:12:01.990637 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:01.990611 5013 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-maas-api-tls" pod "8d62df41-e0d5-4376-912f-f61f8d1fbd2d" (UID: "8d62df41-e0d5-4376-912f-f61f8d1fbd2d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 16:12:02.089383 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:02.089320 5013 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-openshift-service-ca\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:12:02.089383 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:02.089344 5013 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfsm5\" (UniqueName: \"kubernetes.io/projected/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-kube-api-access-qfsm5\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:12:02.089383 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:02.089353 5013 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-maas-api-tls\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:12:02.089383 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:02.089363 5013 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d62df41-e0d5-4376-912f-f61f8d1fbd2d-metrics-certs\") on node \"ip-10-0-129-40.ec2.internal\" DevicePath \"\"" Oct 30 16:12:02.644350 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:02.644318 5013 generic.go:356] "Generic (PLEG): container finished" podID="8d62df41-e0d5-4376-912f-f61f8d1fbd2d" containerID="b3de0a1c1e760905faa28a836fd2efa3b0611a33069b8189e30d6a53945bd793" exitCode=0 Oct 30 16:12:02.644550 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:02.644363 5013 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" Oct 30 16:12:02.644550 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:02.644403 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" event={"ID":"8d62df41-e0d5-4376-912f-f61f8d1fbd2d","Type":"ContainerDied","Data":"b3de0a1c1e760905faa28a836fd2efa3b0611a33069b8189e30d6a53945bd793"} Oct 30 16:12:02.644550 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:02.644460 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-258kv" event={"ID":"8d62df41-e0d5-4376-912f-f61f8d1fbd2d","Type":"ContainerDied","Data":"0dc8d418f5119a5b16f6f6054f9379a1a3573c16d8dc284b03db8db05a0bd90d"} Oct 30 16:12:02.644550 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:02.644483 5013 scope.go:122] "RemoveContainer" containerID="b3de0a1c1e760905faa28a836fd2efa3b0611a33069b8189e30d6a53945bd793" Oct 30 16:12:02.653502 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:02.653482 5013 scope.go:122] "RemoveContainer" containerID="b3de0a1c1e760905faa28a836fd2efa3b0611a33069b8189e30d6a53945bd793" Oct 30 16:12:02.653760 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:12:02.653744 5013 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3de0a1c1e760905faa28a836fd2efa3b0611a33069b8189e30d6a53945bd793\": container with ID starting with b3de0a1c1e760905faa28a836fd2efa3b0611a33069b8189e30d6a53945bd793 not found: ID does not exist" containerID="b3de0a1c1e760905faa28a836fd2efa3b0611a33069b8189e30d6a53945bd793" Oct 30 16:12:02.653839 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:02.653769 5013 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3de0a1c1e760905faa28a836fd2efa3b0611a33069b8189e30d6a53945bd793"} err="failed to get container status \"b3de0a1c1e760905faa28a836fd2efa3b0611a33069b8189e30d6a53945bd793\": rpc error: code = NotFound desc = could not find container \"b3de0a1c1e760905faa28a836fd2efa3b0611a33069b8189e30d6a53945bd793\": container with ID starting with b3de0a1c1e760905faa28a836fd2efa3b0611a33069b8189e30d6a53945bd793 not found: ID does not exist" Oct 30 16:12:02.694229 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:02.694189 5013 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-258kv"] Oct 30 16:12:02.704957 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:02.704933 5013 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-258kv"] Oct 30 16:12:04.005882 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:04.005844 5013 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d62df41-e0d5-4376-912f-f61f8d1fbd2d" path="/var/lib/kubelet/pods/8d62df41-e0d5-4376-912f-f61f8d1fbd2d/volumes" Oct 30 16:12:26.976056 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:12:26.976018 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:12:55.428257 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:55.428223 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-c7x2d_e866b86c-ac64-45a1-af1d-5e6d59aeaa52/manager/0.log" Oct 30 16:12:55.635999 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:55.635965 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-q55hh_072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83/manager/0.log" Oct 30 16:12:55.858371 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:55.858292 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-vwt89_fcb0b34e-a57f-4375-b244-ecdc3f1848c4/manager/0.log" Oct 30 16:12:56.640643 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:56.640602 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-f87fcf56c-pxchz_95519221-ff43-4b09-9ec5-cf98e595d03e/manager/0.log" Oct 30 16:12:56.869053 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:56.869020 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-observability-584f99c788-9zvzt_a9dfd31a-7eb0-4293-a813-be0cdcdea0f8/manager/0.log" Oct 30 16:12:56.976052 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:12:56.975970 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:12:57.114381 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:57.114358 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p_7db72d9d-feff-40d4-997f-560e5f1b1ef1/copy-manifests/0.log" Oct 30 16:12:57.146463 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:57.146420 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p_7db72d9d-feff-40d4-997f-560e5f1b1ef1/manager/0.log" Oct 30 16:12:57.660902 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:57.660851 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:12:57.883665 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:57.883626 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-sdqhw_2e87657e-b3bd-4a34-b354-3c0d0a740fc8/proxy/0.log" Oct 30 16:12:58.683613 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:12:58.683582 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-fmb6s_f05e504c-8b40-44de-ab8b-445f0ac4d80d/maas-api/0.log" Oct 30 16:13:00.733866 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:00.733835 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-tlqhn_20477300-291f-4372-a44a-69547e1b7216/postgres/0.log" Oct 30 16:13:02.766641 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:02.766601 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-kvlxj_350b80bd-337e-4eeb-a491-bcd3e47834cf/manager/0.log" Oct 30 16:13:02.973123 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:02.973096 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-tjbk9_bf99dd84-8640-43ee-8fa8-acb26616da8c/kuadrant-console-plugin/0.log" Oct 30 16:13:04.230780 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:04.230747 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 16:13:04.233635 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:04.233610 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 16:13:05.428901 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:05.428863 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-bl2mn_3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e/istio-proxy/0.log" Oct 30 16:13:05.855941 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:05.855853 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-fljdz_f333930d-c55e-4cff-94b4-85861e2521b8/istio-proxy/0.log" Oct 30 16:13:06.061657 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:06.061629 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-gqsbz_8c23430c-a96b-404c-8772-0fa74158186b/istio-proxy/0.log" Oct 30 16:13:07.717131 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:07.717101 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-779688d9fc-m8g4f_0a1bfd21-ddfa-4820-8356-49214c4cfc3b/router/0.log" Oct 30 16:13:20.058318 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:20.058244 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-c7x2d_e866b86c-ac64-45a1-af1d-5e6d59aeaa52/manager/0.log" Oct 30 16:13:20.260300 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:20.260270 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-q55hh_072400c9-bf8c-42f7-a3f1-2b8bc4b3ac83/manager/0.log" Oct 30 16:13:20.482724 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:20.482697 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-vwt89_fcb0b34e-a57f-4375-b244-ecdc3f1848c4/manager/0.log" Oct 30 16:13:21.162936 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:21.162904 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-f87fcf56c-pxchz_95519221-ff43-4b09-9ec5-cf98e595d03e/manager/0.log" Oct 30 16:13:21.382175 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:21.382141 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-observability-584f99c788-9zvzt_a9dfd31a-7eb0-4293-a813-be0cdcdea0f8/manager/0.log" Oct 30 16:13:21.616655 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:21.616584 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p_7db72d9d-feff-40d4-997f-560e5f1b1ef1/copy-manifests/0.log" Oct 30 16:13:21.649098 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:21.649068 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b7c6fc8bc-7mg6p_7db72d9d-feff-40d4-997f-560e5f1b1ef1/manager/0.log" Oct 30 16:13:22.375505 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:22.375477 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-sdqhw_2e87657e-b3bd-4a34-b354-3c0d0a740fc8/proxy/0.log" Oct 30 16:13:22.949945 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:22.949910 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-fmb6s_f05e504c-8b40-44de-ab8b-445f0ac4d80d/maas-api/0.log" Oct 30 16:13:25.080586 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:25.080551 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-tlqhn_20477300-291f-4372-a44a-69547e1b7216/postgres/0.log" Oct 30 16:13:26.649298 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:26.649268 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-kvlxj_350b80bd-337e-4eeb-a491-bcd3e47834cf/manager/0.log" Oct 30 16:13:26.842950 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:26.842924 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-tjbk9_bf99dd84-8640-43ee-8fa8-acb26616da8c/kuadrant-console-plugin/0.log" Oct 30 16:13:26.975919 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:13:26.975842 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:13:29.096443 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:29.096416 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-bl2mn_3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e/istio-proxy/0.log" Oct 30 16:13:29.510845 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:29.510822 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-fljdz_f333930d-c55e-4cff-94b4-85861e2521b8/istio-proxy/0.log" Oct 30 16:13:29.719852 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:29.719824 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-gqsbz_8c23430c-a96b-404c-8772-0fa74158186b/istio-proxy/0.log" Oct 30 16:13:31.336077 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:31.336042 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-779688d9fc-m8g4f_0a1bfd21-ddfa-4820-8356-49214c4cfc3b/router/0.log" Oct 30 16:13:54.485924 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:54.485894 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:13:56.976239 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:13:56.976186 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:13:57.794169 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:13:57.794072 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:14:26.977211 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:14:26.977162 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:14:56.870811 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:14:56.870728 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:14:56.975103 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:14:56.975071 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:14:59.117239 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:14:59.117186 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:15:01.869649 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:01.869621 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kpfss_bbfe6e96-808d-4fb6-bce1-306f37eaf69d/global-pull-secret-syncer/0.log" Oct 30 16:15:02.054534 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:02.054505 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-8sbdg_c04fe7dc-a0c3-4626-8512-a519e8821681/connection-checker/0.log" Oct 30 16:15:02.181014 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:02.180985 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-cnb75_eee942d8-9866-4315-ab29-d721cf68455c/connection-checker/0.log" Oct 30 16:15:02.315259 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:02.315235 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-s9vdp_2d1df50d-be8d-4d79-ae09-2eb0853f313d/connection-checker/0.log" Oct 30 16:15:02.472894 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:02.472803 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" Oct 30 16:15:02.900829 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:02.900802 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-40.ec2.internal_5ea352a0aff2f2b8b8e81204bcdb2ad9/haproxy/0.log" Oct 30 16:15:07.616184 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:07.616149 5013 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8vnr_7df10531-4deb-4b95-ba47-9fb316775878/konnectivity-agent/0.log" limit=1024 Oct 30 16:15:09.214153 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:09.214124 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-kvlxj_350b80bd-337e-4eeb-a491-bcd3e47834cf/manager/0.log" Oct 30 16:15:09.346886 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:09.346859 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-tjbk9_bf99dd84-8640-43ee-8fa8-acb26616da8c/kuadrant-console-plugin/0.log" Oct 30 16:15:11.059285 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.059233 5013 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5"] Oct 30 16:15:11.059884 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.059866 5013 cpu_manager.go:414] "containerMap: removing container" podUID="8d62df41-e0d5-4376-912f-f61f8d1fbd2d" containerName="maas-api" Oct 30 16:15:11.059927 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.059887 5013 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d62df41-e0d5-4376-912f-f61f8d1fbd2d" containerName="maas-api" Oct 30 16:15:11.060047 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.060037 5013 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d62df41-e0d5-4376-912f-f61f8d1fbd2d" containerName="maas-api" Oct 30 16:15:11.063314 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.063298 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.085171 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.085148 5013 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vkvmm\"/\"default-dockercfg-w6wsh\"" Oct 30 16:15:11.097317 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.097294 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vkvmm\"/\"kube-root-ca.crt\"" Oct 30 16:15:11.097523 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.097499 5013 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vkvmm\"/\"openshift-service-ca.crt\"" Oct 30 16:15:11.099714 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.099688 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5"] Oct 30 16:15:11.176176 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.176148 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/abcccfc6-c4ab-43d0-980c-159c4c240173-lib-modules\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.176176 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.176176 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/abcccfc6-c4ab-43d0-980c-159c4c240173-podres\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.176379 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.176220 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/abcccfc6-c4ab-43d0-980c-159c4c240173-proc\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.176379 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.176330 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvgw5\" (UniqueName: \"kubernetes.io/projected/abcccfc6-c4ab-43d0-980c-159c4c240173-kube-api-access-gvgw5\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.176447 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.176384 5013 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abcccfc6-c4ab-43d0-980c-159c4c240173-sys\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.277475 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.277446 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abcccfc6-c4ab-43d0-980c-159c4c240173-sys\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.277613 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.277511 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/abcccfc6-c4ab-43d0-980c-159c4c240173-lib-modules\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.277613 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.277532 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/abcccfc6-c4ab-43d0-980c-159c4c240173-podres\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.277613 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.277546 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/abcccfc6-c4ab-43d0-980c-159c4c240173-proc\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.277613 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.277564 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abcccfc6-c4ab-43d0-980c-159c4c240173-sys\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.277613 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.277571 5013 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvgw5\" (UniqueName: \"kubernetes.io/projected/abcccfc6-c4ab-43d0-980c-159c4c240173-kube-api-access-gvgw5\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.277772 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.277644 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/abcccfc6-c4ab-43d0-980c-159c4c240173-proc\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.277772 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.277661 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/abcccfc6-c4ab-43d0-980c-159c4c240173-podres\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.277772 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.277647 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/abcccfc6-c4ab-43d0-980c-159c4c240173-lib-modules\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.300713 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.300690 5013 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvgw5\" (UniqueName: \"kubernetes.io/projected/abcccfc6-c4ab-43d0-980c-159c4c240173-kube-api-access-gvgw5\") pod \"perf-node-gather-daemonset-wjgt5\" (UID: \"abcccfc6-c4ab-43d0-980c-159c4c240173\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.376040 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.375980 5013 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:11.681897 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:11.681870 5013 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5"] Oct 30 16:15:11.683887 ip-10-0-129-40 kubenswrapper[5013]: W1030 16:15:11.683854 5013 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podabcccfc6_c4ab_43d0_980c_159c4c240173.slice/crio-ad308d2644d4aea91e390f55296bfc7bb5d8311d57dc6d857d2cc804ab6ff1bc WatchSource:0}: Error finding container ad308d2644d4aea91e390f55296bfc7bb5d8311d57dc6d857d2cc804ab6ff1bc: Status 404 returned error can't find the container with id ad308d2644d4aea91e390f55296bfc7bb5d8311d57dc6d857d2cc804ab6ff1bc Oct 30 16:15:12.263365 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:12.263313 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-xqb8n_e54831c9-77f6-4bce-b8fc-bbfd1838dddd/cluster-monitoring-operator/0.log" Oct 30 16:15:12.523887 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:12.523812 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" event={"ID":"abcccfc6-c4ab-43d0-980c-159c4c240173","Type":"ContainerStarted","Data":"9a66bff8f4ea3fedb5cf413c9734adc476dd252c9dd3baac2b5422a417f6c107"} Oct 30 16:15:12.523887 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:12.523860 5013 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" event={"ID":"abcccfc6-c4ab-43d0-980c-159c4c240173","Type":"ContainerStarted","Data":"ad308d2644d4aea91e390f55296bfc7bb5d8311d57dc6d857d2cc804ab6ff1bc"} Oct 30 16:15:12.524078 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:12.523917 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:12.571539 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:12.571495 5013 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" podStartSLOduration=1.571483399 podStartE2EDuration="1.571483399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 16:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:15:12.569522795 +0000 UTC m=+2228.720419826" watchObservedRunningTime="2026-10-30 16:15:12.571483399 +0000 UTC m=+2228.722380469" Oct 30 16:15:13.835212 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:13.835169 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xmpms_99088f87-9a18-44c4-b844-8c14d7b11100/node-exporter/0.log" Oct 30 16:15:14.000447 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:14.000422 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xmpms_99088f87-9a18-44c4-b844-8c14d7b11100/kube-rbac-proxy/0.log" Oct 30 16:15:14.134721 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:14.134698 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xmpms_99088f87-9a18-44c4-b844-8c14d7b11100/init-textfile/0.log" Oct 30 16:15:18.539401 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:18.539373 5013 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-wjgt5" Oct 30 16:15:18.772242 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:18.772217 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-tstct_7322d7b1-561a-402f-a9bb-111264b04743/networking-console-plugin/0.log" Oct 30 16:15:20.042672 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:20.042641 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/2.log" Oct 30 16:15:20.046707 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:20.046664 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tj9gm_5306d4a8-24eb-461e-9384-f71ed3b41907/console-operator/3.log" Oct 30 16:15:22.807079 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:22.807046 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-xcw5d_ea29f558-cb33-4c42-9663-5db5361081b1/volume-data-source-validator/0.log" Oct 30 16:15:25.572017 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:25.571989 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-bl2mn_3a73d3e0-afac-4a0a-b83c-4d2baaa9db1e/istio-proxy/0.log" Oct 30 16:15:25.605157 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:25.605134 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bbfpq_f77495d2-eb88-48cb-b46e-510fe055471b/dns/0.log" Oct 30 16:15:25.708725 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:25.708694 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-fljdz_f333930d-c55e-4cff-94b4-85861e2521b8/istio-proxy/0.log" Oct 30 16:15:25.774430 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:25.774403 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bbfpq_f77495d2-eb88-48cb-b46e-510fe055471b/kube-rbac-proxy/0.log" Oct 30 16:15:25.804078 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:25.804056 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-gqsbz_8c23430c-a96b-404c-8772-0fa74158186b/istio-proxy/0.log" Oct 30 16:15:26.351409 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:26.351374 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-779688d9fc-m8g4f_0a1bfd21-ddfa-4820-8356-49214c4cfc3b/router/0.log" Oct 30 16:15:26.487122 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:26.487097 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6k2tn_14a2267f-e095-45cb-b0bc-99d7ab20611a/dns-node-resolver/0.log" Oct 30 16:15:26.975632 ip-10-0-129-40 kubenswrapper[5013]: E1030 16:15:26.975589 5013 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g47gl" containerName="extractor" Oct 30 16:15:28.442572 ip-10-0-129-40 kubenswrapper[5013]: I1030 16:15:28.442544 5013 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nvxqq_3173666e-bb4d-4f89-99bd-061c19370b91/node-ca/0.log"